Black Box Variational Inference

12/31/2013
by   Rajesh Ranganath, et al.
0

Variational inference has become a widely used method to approximate posteriors in complex latent variables models. However, deriving a variational inference algorithm generally requires significant model-specific analysis, and these efforts can hinder and deter us from quickly developing and exploring a variety of models for a problem at hand. In this paper, we present a "black box" variational inference algorithm, one that can be quickly applied to many models with little additional derivation. Our method is based on a stochastic optimization of the variational objective where the noisy gradient is computed from Monte Carlo samples from the variational distribution. We develop a number of methods to reduce the variance of the gradient, always maintaining the criterion that we want to avoid difficult model-based derivations. We evaluate our method against the corresponding black box sampling based methods. We find that our method reaches better predictive likelihoods much faster than sampling methods. Finally, we demonstrate that Black Box Variational Inference lets us easily explore a wide space of models by quickly constructing and evaluating several models of longitudinal healthcare data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/03/2016

Overdispersed Black-Box Variational Inference

We introduce overdispersed black-box variational inference, a method to ...
research
11/07/2015

Hierarchical Variational Models

Black box variational inference allows researchers to easily prototype a...
research
12/04/2017

Vprop: Variational Inference using RMSprop

Many computationally-efficient methods for Bayesian deep learning rely o...
research
03/29/2022

Robust, Automated, and Accurate Black-box Variational Inference

Black-box variational inference (BBVI) now sees widespread use in machin...
research
11/10/2015

Black-box α-divergence Minimization

Black-box alpha (BB-α) is a new approximate inference method based on th...
research
10/13/2022

A Dual Control Variate for doubly stochastic optimization and black-box variational inference

In this paper, we aim at reducing the variance of doubly stochastic opti...
research
06/18/2020

Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization

Recent research has seen several advances relevant to black-box VI, but ...

Please sign up or login with your details

Forgot password? Click here to reset