On Convergence and Stability of GANs

05/19/2017
by   Naveen Kodali, et al.
0

We propose studying GAN training dynamics as regret minimization, which is in contrast to the popular view that there is consistent minimization of a divergence between real and generated distributions. We analyze the convergence of GAN training from this new point of view to understand why mode collapse happens. We hypothesize the existence of undesirable local equilibria in this non-convex game to be responsible for mode collapse. We observe that these local equilibria often exhibit sharp gradients of the discriminator function around some real data points. We demonstrate that these degenerate local equilibria can be avoided with a gradient penalty scheme called DRAGAN. We show that DRAGAN enables faster training, achieves improved stability with fewer mode collapses, and leads to generator networks with better modeling performance across a variety of architectures and objective functions.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

page 11

page 12

page 13

page 14

10/23/2017

Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step

Generative adversarial networks (GANs) are a family of generative models...
09/06/2018

GANs beyond divergence minimization

Generative adversarial networks (GANs) can be interpreted as an adversar...
01/13/2018

On the convergence properties of GAN training

Recent work has shown local convergence of GAN training for absolutely c...
10/13/2019

Implicit competitive regularization in GANs

Generative adversarial networks (GANs) are capable of producing high qua...
06/25/2020

Empirical Analysis of Overfitting and Mode Drop in GAN Training

We examine two key questions in GAN training, namely overfitting and mod...
11/07/2017

Theoretical limitations of Encoder-Decoder GAN architectures

Encoder-decoder GANs architectures (e.g., BiGAN and ALI) seek to add an ...
02/08/2019

Mode Collapse and Regularity of Optimal Transportation Maps

This work builds the connection between the regularity theory of optimal...

Code Repositories

DRAGAN

A stable algorithm for GAN training


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.