Geometric Variational Inference
Abstract
Efficiently accessing the information contained in nonlinear and high dimensional probability distributions remains a core challenge in modern statistics. Traditionally, estimators that go beyond point estimates are either categorized as Variational Inference (VI) or MarkovChain MonteCarlo (MCMC) techniques. While MCMC methods that utilize the geometric properties of continuous probability distributions to increase their efficiency have been proposed, VI methods rarely use the geometry. This work aims to fill this gap and proposes geometric Variational Inference (geoVI), a method based on Riemannian geometry and the Fisher information metric. It is used to construct a coordinate transformation that relates the Riemannian manifold associated with the metric to Euclidean space. The distribution, expressed in the coordinate system induced by the transformation, takes a particularly simple form that allows for an accurate variational approximation by a normal distribution. Furthermore, the algorithmic structure allows for an efficient implementation of geoVI which is demonstrated on multiple examples, ranging from lowdimensional illustrative ones to nonlinear, hierarchical Bayesian inverse problems in thousands of dimensions.
 Publication:

Entropy
 Pub Date:
 July 2021
 DOI:
 10.3390/e23070853
 arXiv:
 arXiv:2105.10470
 Bibcode:
 2021Entrp..23..853F
 Keywords:

 Statistics  Methodology;
 Astrophysics  Instrumentation and Methods for Astrophysics;
 Statistics  Machine Learning
 EPrint:
 42 pages, 18 figures, accepted by Entropy