Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning

11/06/2016
by   Dilin Wang, et al.
0

We propose a simple algorithm to train stochastic neural networks to draw samples from given target distributions for probabilistic inference. Our method is based on iteratively adjusting the neural network parameters so that the output changes along a Stein variational gradient that maximumly decreases the KL divergence with the target distribution. Our method works for any target distribution specified by their unnormalized density function, and can train any black-box architectures that are differentiable in terms of the parameters we want to adapt. As an application of our method, we propose an amortized MLE algorithm for training deep energy model, where a neural sampler is adaptively trained to approximate the likelihood function. Our method mimics an adversarial game between the deep energy model and the neural sampler, and obtains realistic-looking images competitive with the state-of-the-art results.

READ FULL TEXT

page 8

page 9

page 10

page 13

research
07/20/2017

Learning to Draw Samples with Amortized Stein Variational Gradient Descent

We propose a simple algorithm to train stochastic neural networks to dra...
research
01/03/2019

Adversarial Learning of a Sampler Based on an Unnormalized Distribution

We investigate adversarial learning in the case when only an unnormalize...
research
03/06/2020

Training Deep Energy-Based Models with f-Divergence Minimization

Deep energy-based models (EBMs) are very flexible in distribution parame...
research
04/03/2023

Non-Generative Energy Based Models

Energy-based models (EBM) have become increasingly popular within comput...
research
04/07/2023

Conservative objective models are a special kind of contrastive divergence-based energy model

In this work we theoretically show that conservative objective models (C...
research
11/10/2015

Black-box α-divergence Minimization

Black-box alpha (BB-α) is a new approximate inference method based on th...
research
02/27/2017

Approximate Inference with Amortised MCMC

We propose a novel approximate inference algorithm that approximates a t...

Please sign up or login with your details

Forgot password? Click here to reset