Equivalence of Optimality Criteria for Markov Decision Process and Model Predictive Control
Abstract
This paper shows that the optimal policy and value functions of a Markov Decision Process (MDP), either discounted or not, can be captured by a finitehorizon undiscounted Optimal Control Problem (OCP), even if based on an inexact model. This can be achieved by selecting a proper stage cost and terminal cost for the OCP. A very useful particular case of OCP is a Model Predictive Control (MPC) scheme where a deterministic (possibly nonlinear) model is used to reduce the computational complexity. This observation leads us to parameterize an MPC scheme fully, including the cost function. In practice, Reinforcement Learning algorithms can then be used to tune the parameterized MPC scheme. We verify the developed theorems analytically in an LQR case and we investigate some other nonlinear examples in simulations.
 Publication:

arXiv eprints
 Pub Date:
 October 2022
 DOI:
 10.48550/arXiv.2210.04302
 arXiv:
 arXiv:2210.04302
 Bibcode:
 2022arXiv221004302B
 Keywords:

 Electrical Engineering and Systems Science  Systems and Control
 EPrint:
 8 pages and 10 figures