Iterative Amortized Inference

07/24/2018
by   Joseph Marino, et al.
2

Inference models are a key component in scaling variational inference to deep latent variable models, most notably as encoder networks in variational auto-encoders (VAEs). By replacing conventional optimization-based inference with a learned model, inference is amortized over data examples and therefore more computationally efficient. However, standard inference models are restricted to direct mappings from data to approximate posterior estimates. The failure of these models to reach fully optimized approximate posterior estimates results in an amortization gap. We aim toward closing this gap by proposing iterative inference models, which learn to perform inference optimization through repeatedly encoding gradients. Our approach generalizes standard inference models in VAEs and provides insight into several empirical findings, including top-down inference techniques. We demonstrate the inference optimization capabilities of iterative inference models and show that they outperform standard inference models on several benchmark data sets of images and text.

READ FULL TEXT

page 3

page 6

page 7

page 20

research
11/13/2018

A General Method for Amortizing Variational Filtering

We introduce the variational filtering EM algorithm, a simple, general-p...
research
11/20/2015

The Variational Gaussian Process

Variational inference is a powerful tool for approximate inference, and ...
research
03/07/2019

The Variational Predictive Natural Gradient

Variational inference transforms posterior inference into parametric opt...
research
07/20/2023

Amortized Variational Inference: When and Why?

Amortized variational inference (A-VI) is a method for approximating the...
research
10/14/2022

On the Relationship Between Variational Inference and Auto-Associative Memory

In this article, we propose a variational inference formulation of auto-...
research
12/01/2017

Faithful Model Inversion Substantially Improves Auto-encoding Variational Inference

In learning deep generative models, the encoder for variational inferenc...
research
11/30/2022

Variational Laplace Autoencoders

Variational autoencoders employ an amortized inference model to approxim...

Please sign up or login with your details

Forgot password? Click here to reset