DeepAI
Log In Sign Up

Efficient Semi-Implicit Variational Inference

01/15/2021
by   Vincent Moens, et al.
10

In this paper, we propose CI-VI an efficient and scalable solver for semi-implicit variational inference (SIVI). Our method, first, maps SIVI's evidence lower bound (ELBO) to a form involving a nonlinear functional nesting of expected values and then develops a rigorous optimiser capable of correctly handling bias inherent to nonlinear nested expectations using an extrapolation-smoothing mechanism coupled with gradient sketching. Our theoretical results demonstrate convergence to a stationary point of the ELBO in general non-convex settings typically arising when using deep network models and an order of O(t^-4/5) gradient-bias-vanishing rate. We believe these results generalise beyond the specific nesting arising from SIVI to other forms. Finally, in a set of experiments, we demonstrate the effectiveness of our algorithm in approximating complex posteriors on various data-sets including those from natural language processing.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/05/2018

Doubly Semi-Implicit Variational Inference

We extend the existing framework of semi-implicit variational inference ...
11/04/2020

Quantized Variational Inference

We present Quantized Variational Inference, a new algorithm for Evidence...
05/28/2018

Semi-Implicit Variational Inference

Semi-implicit variational inference (SIVI) is introduced to expand the c...
02/28/2017

Hierarchical Implicit Models and Likelihood-Free Variational Inference

Implicit probabilistic models are a flexible class of models defined by ...
10/31/2015

Faster Stochastic Variational Inference using Proximal-Gradient Methods with General Divergence Functions

Several recent works have explored stochastic gradient methods for varia...
11/29/2017

On the use of bootstrap with variational inference: Theory, interpretation, and a two-sample test example

Variational inference is a general approach for approximating complex de...
10/31/2020

Hyperbolic Graph Embedding with Enhanced Semi-Implicit Variational Inference

Efficient modeling of relational data arising in physical, social, and i...