A relaxed technical assumption for posterior samplingbased reinforcement learning for control of unknown linear systems
Abstract
We revisit the Thompson sampling algorithm to control an unknown linear quadratic (LQ) system recently proposed by Ouyang et al (arXiv:1709.04047). The regret bound of the algorithm was derived under a technical assumption on the induced norm of the closed loop system. In this technical note, we show that by making a minor modification in the algorithm (in particular, ensuring that an episode does not end too soon), this technical assumption on the induced norm can be replaced by a milder assumption in terms of the spectral radius of the closed loop system. The modified algorithm has the same Bayesian regret of $\tilde{\mathcal{O}}(\sqrt{T})$, where $T$ is the timehorizon and the $\tilde{\mathcal{O}}(\cdot)$ notation hides logarithmic terms in~$T$.
 Publication:

arXiv eprints
 Pub Date:
 August 2021
 DOI:
 10.48550/arXiv.2108.08502
 arXiv:
 arXiv:2108.08502
 Bibcode:
 2021arXiv210808502G
 Keywords:

 Electrical Engineering and Systems Science  Systems and Control;
 Computer Science  Artificial Intelligence;
 Mathematics  Optimization and Control
 EPrint:
 Proc 2022 IEEE Conference on Decision and Control