Fast Variational Inference in the Conjugate Exponential Family

06/22/2012
by   James Hensman, et al.
0

We present a general method for deriving collapsed variational inference algo- rithms for probabilistic models in the conjugate exponential family. Our method unifies many existing approaches to collapsed variational inference. Our collapsed variational inference leads to a new lower bound on the marginal likelihood. We exploit the information geometry of the bound to derive much faster optimization methods based on conjugate gradients for these models. Our approach is very general and is easily applied to any model where the mean field update equations have been derived. Empirically we show significant speed-ups for probabilistic models optimized using our bound.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/03/2023

A Tutorial on Parametric Variational Inference

Variational inference uses optimization, rather than integration, to app...
research
11/04/2020

Quantized Variational Inference

We present Quantized Variational Inference, a new algorithm for Evidence...
research
10/22/2020

Probabilistic Circuits for Variational Inference in Discrete Graphical Models

Inference in discrete graphical models with variational methods is diffi...
research
03/21/2017

A Deterministic Global Optimization Method for Variational Inference

Variational inference methods for latent variable statistical models hav...
research
10/14/2020

Flexible mean field variational inference using mixtures of non-overlapping exponential families

Sparse models are desirable for many applications across diverse domains...
research
08/11/2020

Evidence bounds in singular models: probabilistic and variational perspectives

The marginal likelihood or evidence in Bayesian statistics contains an i...
research
10/14/2015

Embarrassingly Parallel Variational Inference in Nonconjugate Models

We develop a parallel variational inference (VI) procedure for use in da...

Please sign up or login with your details

Forgot password? Click here to reset