Research Overview

My research goal is to develop online planning algorithms and systems that enable physical robots to operate intelligently, safely, and efficiently, with provable guarantees. Currently, my research approaches include:

  1. Devising provable and efficient online algorithms for MDPs and POMDPs with continuous and hybrid spaces using importance sampling and continuous multi-armed bandits.
  2. Integrating online motion planning under uncertainty with computer vision and machine learning to solve realistic motion planning problems with high dimensional observations.
  3. Applying motion planning techniques to other domains including nuclear radiation detection with drones and ecological population dynamics control.

My Google Scholar page is available here.

Academic Papers

Submitted

Voronoi progressive widening (VPW) guides the selection of actions to widen the search tree.

Voronoi Progressive Widening: Efficient Online Solvers for Continuous State, Action, and Observation POMDPs.
Lim, M. H., Tomlin, C. J., & Sunberg, Z. N. (2020). Submitted to CDC 2021.
[arXiv] (Full ver. with Appendix) / [Github]

This paper introduces Voronoi Progressive Widening (VPW), a generalization of Voronoi optimistic optimization (VOO) and action progressive widening to partially observable Markov decision processes (POMDPs). Tree search algorithms can use VPW to effectively handle continuous or hybrid action spaces by efficiently balancing local and global action searching. This paper proposes two VPW-based algorithms and analyzes them from theoretical and simulation perspectives. Voronoi Optimistic Weighted Sparse Sampling (VOWSS) is a theoretical tool that justifies VPW-based online solvers, and it is the first algorithm with global convergence guarantees for continuous state, action, and observation POMDPs. Voronoi Optimistic Monte Carlo Planning with Observation Weighting (VOMCPOW) is a versatile and efficient algorithm that consistently outperforms state-of-the-art POMDP algorithms in several simulation experiments.


Published Works

POWSS tree schematic diagram for continuous observation POMDPs.

Sparse tree search optimality guarantees in POMDPs with continuous observation spaces.
Lim, M. H., Tomlin, C. J., & Sunberg, Z. N. (2020). IJCAI-PRICAI 2020.
[arXiv] (Full ver. with Appendix) / [IJCAI] / [5-min Video]

Partially observable Markov decision processes (POMDPs) with continuous state and observation spaces have powerful flexibility for representing real-world decision and control problems but are notoriously difficult to solve. Recent online sampling-based algorithms that use observation likelihood weighting have shown unprecedented effectiveness in domains with continuous observation spaces. However there has been no formal theoretical justification for this technique. This work offers such a justification, proving that a simplified algorithm, partially observable weighted sparse sampling (POWSS), will estimate Q-values accurately with high probability and can be made to perform arbitrarily near the optimal solution by increasing computational power.


Binder cumulant comparison of thermal and diabatic fit for different ferromagnetic interactions.

Creating analogs of thermal distributions from diabatic excitations in ion-trap-based quantum simulation.
Lim, M. H., Yoshimura, B. T., & Freericks, J. K. (2016). New Journal of Physics 18 043026.
[arXiv] / [Journal]

One broad goal of quantum simulation is to start a simple quantum system in its ground state and slowly evolve the Hamiltonian to a complex one, maintaining the ground state throughout the evolution (called adiabatic state preparation). This provides a natural setting to create a highly entangled and correlated quantum state if the final Hamiltonian supports such a ground state. In ion-trap-based quantum simulations, coherence times are too short to allow for such ground-state evolution for large chains, because the rapid evolution of the system creates excitations to higher energy states. Because the probability for this excitation depends exponentially on the excitation energy and because the thermal distribution also depends exponentially on the excitation energy, we investigate whether this so-called diabatic excitation can create the analog of a thermal distribution; as this could serve as an alternative for creating thermal states of complex quantum systems without requiring contact with a heat bath. In this work, we explore this relationship and determine situations, where diabatic excitation can approximately create thermal states.


Golomb Lombardi

Golomb's graph, an example of a unit distance graph in R2.

On the Chromatic Number of R4.
Exoo, G., Ismailescu, D., & Lim, M. (2014). Springer Series: Discrete Computational Geometry 52(2): 416.
[Journal]

The lower bound for the chromatic number of R4 is improved from 7 to 9. Three graphs with unit distance embeddings in R4 are described. The first is a 7-chromatic graph of order 14 whose chromatic number can be verified by inspection. The second is an 8-chromatic graph of order 26. In this case the chromatic number can be verified quickly by a simple computer program. The third graph is a 9-chromatic graph of order 65 for which computer verification takes about one minute.


Invited Talks and Poster Presentations

The 3rd NorCal Control Workshop (2021)

[Virtual Talk & Poster] POMDPs for Planning in Continuous Spaces.

Two Sigma PhD Research Symposium (2020)

[Invited Talk] Online Robotic Motion Planning Under Uncertainty: POMDPs from Theory to Application

Georgetown Research Experience for Undergraduates Poster Session (2016)

[Poster] Creating analogs of thermal distributions from diabatic excitations in ion-trap-based quantum simulation.

Siemens Regional Finals Competition at Carnegie Mellon University (2013)

[Oral Presentation] On the Chromatic Number of R4.


Industry Research Experience


Awesome Nurobot model present.

nuro.ai (Jun. 2020 - Aug. 2020)
Self-Driving Car Motion Planning

As a software engineer intern, I researched and developed methods to handle occlusions at intersections for online motion planning of a self-driving delivery vehicle.


Some pictures around the IMC Chicago office.

IMC Financial Markets (Jan. 2016 - Aug. 2016)
Quantitative Finance Research

As a summer intern, I modeled inter-index and inter-expiry implied volatility correlations through on-line updates with weighted linear regression. With the model, I proposed a correction to the pricing model to factor in implied vol. correlations to improve estimated trading gains up to $1,000/day.

As a winter intern, I learned theoretical and technical concepts of quantitative trading, such as valuation and quote allocation, through mock trading and mini programming projects using Python's pandas library.