Learning in Implicit Generative Models

10/11/2016
by   Shakir Mohamed, et al.
0

Generative adversarial networks (GANs) provide an algorithmic framework for constructing generative models with several appealing properties: they do not require a likelihood function to be specified, only a generating procedure; they provide samples that are sharp and compelling; and they allow us to harness our knowledge of building highly accurate neural network classifiers. Here, we develop our understanding of GANs with the aim of forming a rich view of this growing area of machine learning---to build connections to the diverse set of statistical thinking on this topic, of which much can be gained by a mutual exchange of ideas. We frame GANs within the wider landscape of algorithms for learning in implicit generative models--models that only specify a stochastic procedure with which to generate data--and relate these ideas to modelling problems in related fields, such as econometrics and approximate Bayesian computation. We develop likelihood-free inference methods and highlight hypothesis testing as a principle for learning in implicit generative models, using which we are able to derive the objective function used by GANs, and many other related objectives. The testing viewpoint directs our focus to the general problem of density ratio estimation. There are four approaches for density ratio estimation, one of which is a solution using classifiers to distinguish real from generated data. Other approaches such as divergence minimisation and moment matching have also been explored in the GAN literature, and we synthesise these views to form an understanding in terms of the relationships between them and the wider literature, highlighting avenues for future exploration and cross-pollination.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/24/2017

Flow-GAN: Bridging implicit and prescribed learning in generative models

Evaluating the performance of generative models for unsupervised learnin...
research
10/10/2016

Generative Adversarial Nets from a Density Ratio Estimation Perspective

Generative adversarial networks (GANs) are successful deep generative mo...
research
09/28/2019

Stein Bridging: Enabling Mutual Reinforcement between Explicit and Implicit Generative Models

Deep generative models are generally categorized into explicit models an...
research
05/19/2017

Gradient Estimators for Implicit Models

Implicit models, which allow for the generation of samples but not for p...
research
02/27/2017

Variational Inference using Implicit Distributions

Generative adversarial networks (GANs) have given us a great tool to fit...
research
09/04/2017

Learning Implicit Generative Models Using Differentiable Graph Tests

Recently, there has been a growing interest in the problem of learning r...
research
10/29/2019

Kernel-Guided Training of Implicit Generative Models with Stability Guarantees

Modern implicit generative models such as generative adversarial network...

Please sign up or login with your details

Forgot password? Click here to reset