Variational Inference in Nonconjugate Models

09/19/2012
by   Chong Wang, et al.
0

Mean-field variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, mean-field methods approximately compute the posterior with a coordinate-ascent optimization algorithm. When the model is conditionally conjugate, the coordinate updates are easily derived and in closed form. However, many models of interest---like the correlated topic model and Bayesian logistic regression---are nonconjuate. In these models, mean-field methods cannot be directly applied and practitioners have had to develop variational algorithms on a case-by-case basis. In this paper, we develop two generic methods for nonconjugate models, Laplace variational inference and delta method variational inference. Our methods have several advantages: they allow for easily derived variational algorithms with a wide class of nonconjugate models; they extend and unify some of the existing algorithms that have been derived for specific models; and they work well on real-world datasets. We studied our methods on the correlated topic model, Bayesian logistic regression, and hierarchical Bayesian logistic regression.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/13/2021

Wide Mean-Field Variational Bayesian Neural Networks Ignore the Data

Variational inference enables approximate posterior inference of the hig...
research
03/24/2023

Particle Mean Field Variational Bayes

The Mean Field Variational Bayes (MFVB) method is one of the most comput...
research
01/03/2023

Introducing Variational Inference in Statistics and Data Science Curriculum

Probabilistic models such as logistic regression, Bayesian classificatio...
research
02/02/2018

An Instability in Variational Inference for Topic Models

Topic models are Bayesian models that are frequently used to capture the...
research
05/31/2022

Easy Variational Inference for Categorical Models via an Independent Binary Approximation

We pursue tractable Bayesian analysis of generalized linear models (GLMs...
research
10/17/2015

A General Method for Robust Bayesian Modeling

Robust Bayesian models are appealing alternatives to standard models, pr...
research
01/28/2021

Low Complexity Approximate Bayesian Logistic Regression for Sparse Online Learning

Theoretical results show that Bayesian methods can achieve lower bounds ...

Please sign up or login with your details

Forgot password? Click here to reset