Algorithms for meanfield variational inference via polyhedral optimization in the Wasserstein space
Abstract
We develop a theory of finitedimensional polyhedral subsets over the Wasserstein space and optimization of functionals over them via firstorder methods. Our main application is to the problem of meanfield variational inference, which seeks to approximate a distribution $\pi$ over $\mathbb{R}^d$ by a product measure $\pi^\star$. When $\pi$ is strongly logconcave and logsmooth, we provide (1) approximation rates certifying that $\pi^\star$ is close to the minimizer $\pi^\star_\diamond$ of the KL divergence over a \emph{polyhedral} set $\mathcal{P}_\diamond$, and (2) an algorithm for minimizing $\text{KL}(\cdot\\pi)$ over $\mathcal{P}_\diamond$ with accelerated complexity $O(\sqrt \kappa \log(\kappa d/\varepsilon^2))$, where $\kappa$ is the condition number of $\pi$.
 Publication:

arXiv eprints
 Pub Date:
 December 2023
 DOI:
 10.48550/arXiv.2312.02849
 arXiv:
 arXiv:2312.02849
 Bibcode:
 2023arXiv231202849J
 Keywords:

 Mathematics  Statistics Theory;
 Computer Science  Machine Learning;
 Mathematics  Optimization and Control
 EPrint:
 40 pages