Relay Variational Inference: A Method for Accelerated Encoderless VI

10/26/2021
by   Amir Zadeh, et al.
33

Variational Inference (VI) offers a method for approximating intractable likelihoods. In neural VI, inference of approximate posteriors is commonly done using an encoder. Alternatively, encoderless VI offers a framework for learning generative models from data without encountering suboptimalities caused by amortization via an encoder (e.g. in presence of missing or uncertain data). However, in absence of an encoder, such methods often suffer in convergence due to the slow nature of gradient steps required to learn the approximate posterior parameters. In this paper, we introduce Relay VI (RVI), a framework that dramatically improves both the convergence and performance of encoderless VI. In our experiments over multiple datasets, we study the effectiveness of RVI in terms of convergence speed, loss, representation power and missing data imputation. We find RVI to be a unique tool, often superior in both performance and convergence speed to previously proposed encoderless as well as amortized VI models (e.g. VAE).

READ FULL TEXT

page 8

page 11

research
06/06/2019

Amortized Inference of Variational Bounds for Learning Noisy-OR

Classical approaches for approximate inference depend on cleverly design...
research
08/05/2017

Boosting Variational Inference: an Optimization Perspective

Variational Inference is a popular technique to approximate a possibly i...
research
03/03/2019

Variational Auto-Decoder: Neural Generative Modeling from Partial Data

Learning a generative model from partial data (data with missingness) is...
research
03/03/2019

Variational Auto-Decoder

Learning a generative model from partial data (data with missingness) is...
research
12/19/2019

Pseudo-Encoded Stochastic Variational Inference

Posterior inference in directed graphical models is commonly done using ...
research
12/01/2017

Faithful Model Inversion Substantially Improves Auto-encoding Variational Inference

In learning deep generative models, the encoder for variational inferenc...

Please sign up or login with your details

Forgot password? Click here to reset