Variational Bayesian Inference with Stochastic Search

06/27/2012
by   John Paisley, et al.
0

Mean-field variational inference is a method for approximate Bayesian posterior inference. It approximates a full posterior distribution with a factorized set of distributions by maximizing a lower bound on the marginal likelihood. This requires the ability to integrate a sum of terms in the log joint likelihood using this factorized distribution. Often not all integrals are in closed form, which is typically handled by using a lower bound. We present an alternative algorithm based on stochastic optimization that allows for direct optimization of the variational lower bound. This method uses control variates to reduce the variance of the stochastic search gradient, in which existing lower bounds can play an important role. We demonstrate the approach on two non-conjugate models: logistic regression and an approximation to the HDP.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/21/2019

Tutorial: Deriving the Standard Variational Autoencoder (VAE) Loss Function

In Bayesian machine learning, the posterior distribution is typically co...
research
06/28/2016

Automatic Variational ABC

Approximate Bayesian Computation (ABC) is a framework for performing lik...
research
11/22/2010

Variational approximation for heteroscedastic linear models and matching pursuit algorithms

Modern statistical applications involving large data sets have focused a...
research
09/30/2019

Tightening Bounds for Variational Inference by Revisiting Perturbation Theory

Variational inference has become one of the most widely used methods in ...
research
04/06/2015

Early Stopping is Nonparametric Variational Inference

We show that unconverged stochastic gradient descent can be interpreted ...
research
09/27/2018

On some variance reduction properties of the reparameterization trick

The so-called reparameterization trick is widely used in variational inf...
research
10/26/2021

Online Variational Filtering and Parameter Learning

We present a variational method for online state estimation and paramete...

Please sign up or login with your details

Forgot password? Click here to reset