Automatic Variational Inference in Stan

06/10/2015
by   Alp Kucukelbir, et al.
0

Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult to automate. We propose an automatic variational inference algorithm, automatic differentiation variational inference (ADVI). The user only provides a Bayesian model and a dataset; nothing else. We make no conjugacy assumptions and support a broad class of models. The algorithm automatically determines an appropriate variational family and optimizes the variational objective. We implement ADVI in Stan (code available now), a probabilistic programming framework. We compare ADVI to MCMC sampling across hierarchical generalized linear models, nonconjugate matrix factorization, and a mixture model. We train the mixture model on a quarter million images. With ADVI we can use variational inference on any model we write in Stan.

READ FULL TEXT
research
03/02/2016

Automatic Differentiation Variational Inference

Probabilistic modeling is iterative. A scientist posits a simple model, ...
research
05/11/2019

Variational inference for neural network matrix factorization and its application to stochastic blockmodeling

We consider the probabilistic analogue to neural network matrix factoriz...
research
05/23/2019

Variational Inference with Mixture Model Approximation: Robotic Applications

We propose a method to approximate the distribution of robot configurati...
research
10/27/2016

Operator Variational Inference

Variational inference is an umbrella term for algorithms which cast Baye...
research
05/19/2022

Variational Inference for Bayesian Bridge Regression

We study the implementation of Automatic Differentiation Variational inf...
research
05/31/2022

Easy Variational Inference for Categorical Models via an Independent Binary Approximation

We pursue tractable Bayesian analysis of generalized linear models (GLMs...
research
01/13/2017

Truncation-free Hybrid Inference for DPMM

Dirichlet process mixture models (DPMM) are a cornerstone of Bayesian no...

Please sign up or login with your details

Forgot password? Click here to reset