Efficient Debiased Variational Bayes by Multilevel Monte Carlo Methods

01/14/2020
by   Kei Ishikawa, et al.
8

Variational Bayes is a method to find a good approximation of the posterior probability distribution of latent variables from a parametric family of distributions. The evidence lower bound (ELBO), which is nothing but the model evidence minus the Kullback-Leibler divergence, has been commonly used as a quality measure in the optimization process. However, the model evidence itself has been considered computationally intractable since it is expressed as a nested expectation with an outer expectation with respect to the training dataset and an inner conditional expectation with respect to latent variables. Similarly, if the Kullback-Leibler divergence is replaced with another divergence metric, the corresponding lower bound on the model evidence is often given by such a nested expectation. The standard (nested) Monte Carlo method can be used to estimate such quantities, whereas the resulting estimate is biased and the variance is often quite large. Recently the authors provided an unbiased estimator of the model evidence with small variance by applying the idea from multilevel Monte Carlo (MLMC) methods. In this article, we give more examples involving nested expectations in the context of variational Bayes where MLMC methods can help construct low-variance unbiased estimators, and provide numerical results which demonstrate the effectiveness of our proposed estimators.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/04/2022

Constructing unbiased gradient estimators with finite variance for conditional stochastic optimization

We study stochastic gradient descent for solving conditional stochastic ...
research
09/02/2019

Multilevel Monte Carlo estimation of the expected value of sample information

We study Monte Carlo estimation of the expected value of sample informat...
research
04/11/2022

Unbiased Multilevel Monte Carlo methods for intractable distributions: MLMC meets MCMC

Constructing unbiased estimators from Markov chain Monte Carlo (MCMC) ou...
research
09/27/2021

Unbiased MLMC-based variational Bayes for likelihood-free inference

Variational Bayes (VB) is a popular tool for Bayesian inference in stati...
research
02/16/2021

Analysis of nested multilevel Monte Carlo using approximate Normal random variables

The multilevel Monte Carlo (MLMC) method has been used for a wide variet...
research
11/01/2016

Variational Inference via χ-Upper Bound Minimization

Variational inference (VI) is widely used as an efficient alternative to...
research
12/11/2019

Sub-sampling and other considerations for efficient risk estimation in large portfolios

Computing risk measures of a financial portfolio comprising thousands of...

Please sign up or login with your details

Forgot password? Click here to reset