A multilevel approach for stochastic nonlinear optimal control
Abstract
We consider a class of finite-time horizon nonlinear stochastic optimal control problem. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging task. We propose a new Monte Carlo approach that significantly improves upon existing methodology. We tackle the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state-space model, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce the cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with ? mean squared error with a cost of ? . In contrast, a cost of ? is required for the existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples.
- Publication:
-
International Journal of Control
- Pub Date:
- May 2022
- DOI:
- 10.1080/00207179.2020.1849805
- arXiv:
- arXiv:1901.05583
- Bibcode:
- 2022IJC....95.1290J
- Keywords:
-
- Mathematics - Optimization and Control;
- Mathematics - Numerical Analysis
- E-Print:
- International Journal of Control, Volume 95, Issue 5, pages: 1290-1304, 2022