A Hierarchical Twotier Approach to Hyperparameter Optimization in Reinforcement Learning
Abstract
Optimization of hyperparameters in reinforcement learning (RL) algorithms is a key task, because they determine how the agent will learn its policy by interacting with its environment, and thus what data is gathered. In this work, an approach that uses Bayesian optimization to perform a twostep optimization is proposed: first, categorical RL structure hyperparameters are taken as binary variables and optimized with an acquisition function tailored for such variables. Then, at a lower level of abstraction, solutionlevel hyperparameters are optimized by resorting to the expected improvement acquisition function, while using the best categorical hyperparameters found in the optimization at the upperlevel of abstraction. This twotier approach is validated in a simulated control task. Results obtained are promising and open the way for more userindependent applications of reinforcement learning.
 Publication:

arXiv eprints
 Pub Date:
 September 2019
 arXiv:
 arXiv:1909.08332
 Bibcode:
 2019arXiv190908332C
 Keywords:

 Computer Science  Machine Learning;
 Computer Science  Artificial Intelligence;
 Statistics  Machine Learning
 EPrint:
 Short paper presented in the Jornadas Argentinas de Inform\'atica (JAIIO) 2019 (Salta, Argentina), describing an ongoing research on RL hyperparameter tuning