Markovian Score Climbing: Variational Inference with KL(p||q)

03/23/2020
by   Christian A. Naesseth, et al.
5

Modern variational inference (VI) uses stochastic gradients to avoid intractable expectations, enabling large-scale probabilistic inference in complex models. VI posits a family of approximating distributions q and then finds the member of that family that is closest to the exact posterior p. Traditionally, VI algorithms minimize the "exclusive KL" KL(qp), often for computational convenience. Recent research, however, has also focused on the "inclusive KL" KL(pq), which has good statistical properties that makes it more appropriate for certain inference problems. This paper develops a simple algorithm for reliably minimizing the inclusive KL. Consider a valid MCMC method, a Markov chain whose stationary distribution is p. The algorithm we develop iteratively samples the chain z[k], and then uses those samples to follow the score function of the variational approximation, ∇log q(z[k]) with a Robbins-Monro step-size schedule. This method, which we call Markovian score climbing (MSC), converges to a local optimum of the inclusive KL. It does not suffer from the systematic errors inherent in existing methods, such as Reweighted Wake-Sleep and Neural Adaptive Sequential Monte Carlo, which lead to bias in their final estimates. In a variant that ties the variational approximation directly to the Markov chain, MSC further provides a new algorithm that melds VI and MCMC. We illustrate convergence on a toy model and demonstrate the utility of MSC on Bayesian probit regression for classification as well as a stochastic volatility model for financial data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/10/2019

A Contrastive Divergence for Combining Variational Inference and MCMC

We develop a method to combine Markov chain Monte Carlo (MCMC) and varia...
research
03/07/2023

Variational Inference for Neyman-Scott Processes

Neyman-Scott processes (NSPs) have been applied across a range of fields...
research
02/03/2022

Transport Score Climbing: Variational Inference Using Forward KL and Adaptive Neural Transport

Variational inference often minimizes the "reverse" Kullbeck-Leibler (KL...
research
04/13/2021

The computational asymptotics of Gaussian variational inference

Variational inference is a popular alternative to Markov chain Monte Car...
research
02/04/2021

Variational Inference for Deblending Crowded Starfields

In the image data collected by astronomical surveys, stars and galaxies ...
research
01/15/2020

Newtonian Monte Carlo: single-site MCMC meets second-order gradient methods

Single-site Markov Chain Monte Carlo (MCMC) is a variant of MCMC in whic...
research
04/14/2016

Variational inference for rare variant detection in deep, heterogeneous next-generation sequencing data

The detection of rare variants is important for understanding the geneti...

Please sign up or login with your details

Forgot password? Click here to reset