Semi-Implicit Variational Inference

05/28/2018
by   Mingzhang Yin, et al.
0

Semi-implicit variational inference (SIVI) is introduced to expand the commonly used analytic variational distribution family, by mixing the variational parameter with a flexible distribution. This mixing distribution can assume any density function, explicit or not, as long as independent random samples can be generated via reparameterization. Not only does SIVI expand the variational family to incorporate highly flexible variational distributions, including implicit ones that have no analytic density functions, but also sandwiches the evidence lower bound (ELBO) between a lower bound and an upper bound, and further derives an asymptotically exact surrogate ELBO that is amenable to optimization via stochastic gradient ascent. With a substantially expanded variational family and a novel optimization algorithm, SIVI is shown to closely match the accuracy of MCMC in inferring the posterior in a variety of Bayesian inference tasks.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 15

10/05/2018

Doubly Semi-Implicit Variational Inference

We extend the existing framework of semi-implicit variational inference ...
05/08/2019

Importance Weighted Hierarchical Variational Inference

Variational Inference is a powerful tool in the Bayesian modeling toolki...
10/16/2018

Metropolis-Hastings view on variational inference and adversarial training

In this paper we propose to view the acceptance rate of the Metropolis-H...
01/15/2021

Efficient Semi-Implicit Variational Inference

In this paper, we propose CI-VI an efficient and scalable solver for sem...
11/01/2017

Stochastic Variational Inference for Fully Bayesian Sparse Gaussian Process Regression Models

This paper presents a novel variational inference framework for deriving...
08/26/2019

Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs

A framework to boost efficiency of Bayesian inference in probabilistic p...
08/23/2020

Blindness of score-based methods to isolated components and mixing proportions

A large family of score-based methods are developed recently to solve un...

Code Repositories

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.