Decision-Making with Auto-Encoding Variational Bayes

02/17/2020
by   Romain Lopez, et al.
9

To make decisions based on a model fit by Auto-Encoding Variational Bayes (AEVB), practitioners typically use importance sampling to estimate a functional of the posterior distribution. The variational distribution found by AEVB serves as the proposal distribution for importance sampling. However, this proposal distribution may give unreliable (high variance) importance sampling estimates, thus leading to poor decisions. We explore how changing the objective function for learning the variational distribution, while continuing to learn the generative model based on the ELBO, affects the quality of downstream decisions. For a particular model, we characterize the error of importance sampling as a function of posterior variance and show that proposal distributions learned with evidence upper bounds are better. Motivated by these theoretical results, we propose a novel variant of the VAE. In addition to experimenting with MNIST, we present a full-fledged application of the proposed method to single-cell RNA sequencing. In this challenging instance of multiple hypothesis testing, the proposed method surpasses the current state of the art.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/18/2017

Stein Variational Adaptive Importance Sampling

We propose a novel adaptive importance sampling algorithm which incorpor...
research
05/25/2018

Accurate Computation of Marginal Data Densities Using Variational Bayes

Bayesian model selection and model averaging rely on estimates of margin...
research
09/30/2022

Learning with MISELBO: The Mixture Cookbook

Mixture models in variational inference (VI) is an active field of resea...
research
06/04/2010

Variational Program Inference

We introduce a framework for representing a variety of interesting probl...
research
02/07/2018

Yes, but Did It Work?: Evaluating Variational Inference

While it's always possible to compute a variational approximation to a p...
research
02/09/2023

An information-theoretic learning model based on importance sampling

A crucial assumption underlying the most current theory of machine learn...
research
06/30/2021

Monte Carlo Variational Auto-Encoders

Variational auto-encoders (VAE) are popular deep latent variable models ...

Please sign up or login with your details

Forgot password? Click here to reset