Log In Sign Up

Variational Inference with Locally Enhanced Bounds for Hierarchical Models

by   Tomas Geffner, et al.

Hierarchical models represent a challenging setting for inference algorithms. MCMC methods struggle to scale to large models with many local variables and observations, and variational inference (VI) may fail to provide accurate approximations due to the use of simple variational families. Some variational methods (e.g. importance weighted VI) integrate Monte Carlo methods to give better accuracy, but these tend to be unsuitable for hierarchical models, as they do not allow for subsampling and their performance tends to degrade for high dimensional models. We propose a new family of variational bounds for hierarchical models, based on the application of tightening methods (e.g. importance weighting) separately for each group of local random variables. We show that our approach naturally allows the use of subsampling to get unbiased gradients, and that it fully leverages the power of methods that build tighter lower bounds by applying them independently in lower dimensional spaces, leading to better results and more accurate posterior approximations than relevant baselines.


page 1

page 2

page 3

page 4


A Divergence Bound for Hybrids of MCMC and Variational Inference and an Application to Langevin Dynamics and SGVI

Two popular classes of methods for approximate inference are Markov chai...

Amortized Variational Inference for Simple Hierarchical Models

It is difficult to use subsampling with variational inference in hierarc...

Practical Posterior Error Bounds from Variational Objectives

Variational inference has become an increasingly attractive, computation...

Gauged Mini-Bucket Elimination for Approximate Inference

Computing the partition function Z of a discrete graphical model is a fu...

Importance Weighting and Varational Inference

Recent work used importance sampling ideas for better variational bounds...

Variational Particle Approximations

Approximate inference in high-dimensional, discrete probabilistic models...

Embarrassingly Parallel Variational Inference in Nonconjugate Models

We develop a parallel variational inference (VI) procedure for use in da...