Understanding and Stabilizing GANs' Training Dynamics with Control Theory

09/29/2019
by   Kun Xu, et al.
0

Generative adversarial networks (GANs) have made significant progress on realistic image generation but often suffer from instability during the training process. Most previous analyses mainly focus on the equilibrium that GANs achieve, whereas a gap exists between such theoretical analyses and practical implementations, where it is the training dynamics that plays a vital role in the convergence and stability of GANs. In this paper, we directly model the dynamics of GANs and adopt the control theory to understand and stabilize it. Specifically, we interpret the training process of various GANs as certain types of dynamics in a unified perspective of control theory which enables us to model the stability and convergence easily. Borrowed from control theory, we adopt the widely-used negative feedback control to stabilize the training dynamics, which can be considered as an L2 regularization on the output of the discriminator. We empirically verify our method on both synthetic data and natural image datasets. The results demonstrate that our method can stabilize the training dynamics as well as converge better than baselines.

READ FULL TEXT
research
06/18/2023

Stabilizing GANs' Training with Brownian Motion Controller

The training process of generative adversarial networks (GANs) is unstab...
research
08/19/2020

Direct Adversarial Training for GANs

There is an interesting discovery that several neural networks are vulne...
research
01/28/2018

Improved Training of Generative Adversarial Networks Using Representative Features

Despite of the success of Generative Adversarial Networks (GANs) for ima...
research
07/04/2023

Stability Analysis Framework for Particle-based Distance GANs with Wasserstein Gradient Flow

In this paper, we investigate the training process of generative network...
research
05/22/2018

A Solvable High-Dimensional Model of GAN

Despite the remarkable successes of generative adversarial networks (GAN...
research
08/07/2021

Approximate Last Iterate Convergence in Overparameterized GANs

In this work, we showed that the Implicit Update and Predictive Methods ...
research
05/22/2020

Host-Pathongen Co-evolution Inspired Algorithm Enables Robust GAN Training

Generative adversarial networks (GANs) are pairs of artificial neural ne...

Please sign up or login with your details

Forgot password? Click here to reset