Embarrassingly Parallel Variational Inference in Nonconjugate Models

10/14/2015
by   Willie Neiswanger, et al.
0

We develop a parallel variational inference (VI) procedure for use in data-distributed settings, where each machine only has access to a subset of data and runs VI independently, without communicating with other machines. This type of "embarrassingly parallel" procedure has recently been developed for MCMC inference algorithms; however, in many cases it is not possible to directly extend this procedure to VI methods without requiring certain restrictive exponential family conditions on the form of the model. Furthermore, most existing (nonparallel) VI methods are restricted to use on conditionally conjugate models, which limits their applicability. To combat these issues, we make use of the recently proposed nonparametric VI to facilitate an embarrassingly parallel VI procedure that can be applied to a wider scope of models, including to nonconjugate models. We derive our embarrassingly parallel VI algorithm, analyze our method theoretically, and demonstrate our method empirically on a few nonconjugate models.

READ FULL TEXT
research
06/22/2012

Fast Variational Inference in the Conjugate Exponential Family

We present a general method for deriving collapsed variational inference...
research
10/04/2014

Gamma Processes, Stick-Breaking, and Variational Inference

While most Bayesian nonparametric models in machine learning have focuse...
research
03/18/2019

Approximating exponential family models (not single distributions) with a two-network architecture

Recently much attention has been paid to deep generative models, since t...
research
06/07/2019

Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations

Natural-gradient methods enable fast and simple algorithms for variation...
research
02/11/2022

Bernstein Flows for Flexible Posteriors in Variational Bayes

Variational inference (VI) is a technique to approximate difficult to co...
research
03/08/2022

Variational Inference with Locally Enhanced Bounds for Hierarchical Models

Hierarchical models represent a challenging setting for inference algori...
research
05/16/2017

Learning Hard Alignments with Variational Inference

There has recently been significant interest in hard attention models fo...

Please sign up or login with your details

Forgot password? Click here to reset