High-Dimensional Bayesian Optimization via Nested Riemannian Manifolds
Abstract
Despite the recent success of Bayesian optimization (BO) in a variety of applications where sample efficiency is imperative, its performance may be seriously compromised in settings characterized by high-dimensional parameter spaces. A solution to preserve the sample efficiency of BO in such problems is to introduce domain knowledge into its formulation. In this paper, we propose to exploit the geometry of non-Euclidean search spaces, which often arise in a variety of domains, to learn structure-preserving mappings and optimize the acquisition function of BO in low-dimensional latent spaces. Our approach, built on Riemannian manifolds theory, features geometry-aware Gaussian processes that jointly learn a nested-manifold embedding and a representation of the objective function in the latent space. We test our approach in several benchmark artificial landscapes and report that it not only outperforms other high-dimensional BO approaches in several settings, but consistently optimizes the objective functions, as opposed to geometry-unaware BO methods.
- Publication:
-
arXiv e-prints
- Pub Date:
- October 2020
- DOI:
- 10.48550/arXiv.2010.10904
- arXiv:
- arXiv:2010.10904
- Bibcode:
- 2020arXiv201010904J
- Keywords:
-
- Computer Science - Machine Learning;
- Mathematics - Optimization and Control
- E-Print:
- Accepted for publication in NeurIPS 2020. Code available at https://github.com/NoemieJaquier/GaBOtorch . 13 pages + 5 appendices pages, 5 figures