DeepAI AI Chat
Log In Sign Up

Geometric variational inference

by   Philipp Frank, et al.

Efficiently accessing the information contained in non-linear and high dimensional probability distributions remains a core challenge in modern statistics. Traditionally, estimators that go beyond point estimates are either categorized as Variational Inference (VI) or Markov-Chain Monte-Carlo (MCMC) techniques. While MCMC methods that utilize the geometric properties of continuous probability distributions to increase their efficiency have been proposed, VI methods rarely use the geometry. This work aims to fill this gap and proposes geometric Variational Inference (geoVI), a method based on Riemannian geometry and the Fisher information metric. It is used to construct a coordinate transformation that relates the Riemannian manifold associated with the metric to Euclidean space. The distribution, expressed in the coordinate system induced by the transformation, takes a particularly simple form that allows for an accurate variational approximation by a normal distribution. Furthermore, the algorithmic structure allows for an efficient implementation of geoVI which is demonstrated on multiple examples, ranging from low-dimensional illustrative ones to non-linear, hierarchical Bayesian inverse problems in thousands of dimensions.


page 5

page 18

page 27

page 28

page 29

page 31

page 32


Lagrangian Manifold Monte Carlo on Monge Patches

The efficiency of Markov Chain Monte Carlo (MCMC) depends on how the und...

Learning Model Reparametrizations: Implicit Variational Inference by Fitting MCMC distributions

We introduce a new algorithm for approximate inference that combines rep...

MCMC-Interactive Variational Inference

Leveraging well-established MCMC strategies, we propose MCMC-interactive...

A Variational Inference Framework for Inverse Problems

We present a framework for fitting inverse problem models via variationa...

Manifold lifting: scaling MCMC to the vanishing noise regime

Standard Markov chain Monte Carlo methods struggle to explore distributi...

Clustering in Hilbert simplex geometry

Clustering categorical distributions in the probability simplex is a fun...

AdaAnn: Adaptive Annealing Scheduler for Probability Density Approximation

Approximating probability distributions can be a challenging task, parti...