My research goal is to develop online planning algorithms and systems that enable physical robots to operate intelligently, safely, and efficiently, with provable guarantees. Currently, my research trajectory consists of two general approaches:
1. My research aims to develop versatile online MDP and POMDP algorithms that tackle continuous spaces, which combine provable theoretical convergence guarantees and computational efficiency.
2. My research attempts to integrate online planning under uncertainty with cutting edge principles and techniques from robust optimal control, computer vision and machine learning to enable agents to navigate through more realistic environments.
My Google Scholar page is available here.
Voronoi progressive widening (VPW) guides the selection of actions to widen the search tree.
Voronoi Progressive Widening: Efficient Online Solvers for Continuous Space MDPs and POMDPs with Provably Optimal Components.
Lim, M. H., Tomlin, C. J., & Sunberg, Z. N. (2020). Submitted to ICAPS 2021.
[arXiv] (Full ver. with Appendix) / [Github]
Markov decision processes (MDPs) and partially observable MDPs (POMDPs) can effectively represent complex real-world decision and control problems. However, continuous space MDPs and POMDPs, i.e. those having continuous state, action and observation spaces, are extremely difficult to solve, and there are few online algorithms with convergence guarantees. This paper introduces Voronoi Progressive Widening (VPW), a general technique to modify tree search algorithms to effectively handle continuous or hybrid action spaces, and proposes and evaluates three continuous space solvers: VOSS, VOWSS, and VOMCPOW. VOSS and VOWSS are theoretical tools based on sparse sampling and Voronoi optimistic optimization designed to justify VPW-based online solvers. While previous algorithms have enjoyed convergence guarantees for problems with continuous state and observation spaces, VOWSS is the first with global convergence guarantees for problems that additionally have continuous action spaces. VOMCPOW is a versatile and efficient VPW-based algorithm that consistently outperforms POMCPOW and BOMCP in several simulation experiments.
POWSS tree schematic diagram for continuous observation POMDPs.
Sparse tree search optimality guarantees in POMDPs with continuous observation spaces.
Lim, M. H., Tomlin, C. J., & Sunberg, Z. N. (2020). IJCAI-PRICAI 2020.
[arXiv] (Full ver. with Appendix) / [IJCAI] / [5-min Video]
Partially observable Markov decision processes (POMDPs) with continuous state and observation spaces have powerful flexibility for representing real-world decision and control problems but are notoriously difficult to solve. Recent online sampling-based algorithms that use observation likelihood weighting have shown unprecedented effectiveness in domains with continuous observation spaces. However there has been no formal theoretical justification for this technique. This work offers such a justification, proving that a simplified algorithm, partially observable weighted sparse sampling (POWSS), will estimate Q-values accurately with high probability and can be made to perform arbitrarily near the optimal solution by increasing computational power.
Binder cumulant comparison of thermal and diabatic fit for different ferromagnetic interactions.
Creating analogs of thermal distributions from diabatic excitations in ion-trap-based quantum simulation.
Lim, M. H., Yoshimura, B. T., & Freericks, J. K. (2016). New Journal of Physics 18 043026.
[arXiv] / [Journal]
One broad goal of quantum simulation is to start a simple quantum system in its ground state and slowly evolve the Hamiltonian to a complex one, maintaining the ground state throughout the evolution (called adiabatic state preparation). This provides a natural setting to create a highly entangled and correlated quantum state if the final Hamiltonian supports such a ground state. In ion-trap-based quantum simulations, coherence times are too short to allow for such ground-state evolution for large chains, because the rapid evolution of the system creates excitations to higher energy states. Because the probability for this excitation depends exponentially on the excitation energy and because the thermal distribution also depends exponentially on the excitation energy, we investigate whether this so-called diabatic excitation can create the analog of a thermal distribution; as this could serve as an alternative for creating thermal states of complex quantum systems without requiring contact with a heat bath. In this work, we explore this relationship and determine situations, where diabatic excitation can approximately create thermal states.
On the Chromatic Number of R4.
Exoo, G., Ismailescu, D., & Lim, M. (2014). Springer Series: Discrete Computational Geometry 52(2): 416.
The lower bound for the chromatic number of R4 is improved from 7 to 9. Three graphs with unit distance embeddings in R4 are described. The first is a 7-chromatic graph of order 14 whose chromatic number can be verified by inspection. The second is an 8-chromatic graph of order 26. In this case the chromatic number can be verified quickly by a simple computer program. The third graph is a 9-chromatic graph of order 65 for which computer verification takes about one minute.
The 3rd NorCal Control Workshop (2021)
[Virtual Talk & Poster] POMDPs for Planning in Continuous Spaces.
Two Sigma PhD Research Symposium (2020)
[Invited Talk] Online Robotic Motion Planning Under Uncertainty: POMDPs from Theory to Application
Georgetown Research Experience for Undergraduates Poster Session (2016)
[Poster] Creating analogs of thermal distributions from diabatic excitations in ion-trap-based quantum simulation.
Siemens Regional Finals Competition at Carnegie Mellon University (2013)
[Oral Presentation] On the Chromatic Number of R4.
Awesome Nurobot model present.
nuro.ai (Jun. 2020 - Aug. 2020)
Self-Driving Car Motion Planning
As a software engineer intern, I researched and developed methods to handle occlusions at intersections for online motion planning of a self-driving delivery vehicle.
Some pictures around the IMC Chicago office.
IMC Financial Markets (Jan. 2016 - Aug. 2016)
Quantitative Finance Research
As a summer intern, I modeled inter-index and inter-expiry implied volatility correlations through on-line updates with weighted linear regression. With the model, I proposed a correction to the pricing model to factor in implied vol. correlations to improve estimated trading gains up to $1,000/day.
As a winter intern, I learned theoretical and technical concepts of quantitative trading, such as valuation and quote allocation, through mock trading and mini programming projects using Python's pandas library.