DeepAI AI Chat
Log In Sign Up

Instability and Local Minima in GAN Training with Kernel Discriminators

by   Evan Becker, et al.

Generative Adversarial Networks (GANs) are a widely-used tool for generative modeling of complex data. Despite their empirical success, the training of GANs is not fully understood due to the min-max optimization of the generator and discriminator. This paper analyzes these joint dynamics when the true samples, as well as the generated samples, are discrete, finite sets, and the discriminator is kernel-based. A simple yet expressive framework for analyzing training called the Isolated Points Model is introduced. In the proposed model, the distance between true samples greatly exceeds the kernel width, so each generated point is influenced by at most one true point. Our model enables precise characterization of the conditions for convergence, both to good and bad minima. In particular, the analysis explains two common failure modes: (i) an approximate mode collapse and (ii) divergence. Numerical simulations are provided that predictably replicate these behaviors.


page 8

page 24

page 26


Local Convergence of Gradient Descent-Ascent for Training Generative Adversarial Networks

Generative Adversarial Networks (GANs) are a popular formulation to trai...

Effective Dynamics of Generative Adversarial Networks

Generative adversarial networks (GANs) are a class of machine-learning m...

WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points

Generative adversarial networks (GAN) are a widely used class of deep ge...

Mixed batches and symmetric discriminators for GAN training

Generative adversarial networks (GANs) are pow- erful generative models ...

KGAN: How to Break The Minimax Game in GAN

Generative Adversarial Networks (GANs) were intuitively and attractively...

Towards a Better Global Loss Landscape of GANs

Understanding of GAN training is still very limited. One major challenge...

A Neural Tangent Kernel Perspective of GANs

Theoretical analyses for Generative Adversarial Networks (GANs) generall...