Regularization And Normalization For Generative Adversarial Networks: A Review

08/19/2020
by   Ziqiang Li, et al.
0

Generative adversarial networks(GANs) is a popular generative model. With the development of the deep network, its application is more and more widely. By now, people think that the training of GANs is a two-person zero-sum game(discriminator and generator). The lack of strong supervision information makes the training very difficult, such as non-convergence, mode collapses, gradient disappearance, and the sensitivity of hyperparameters. As we all know, regularization and normalization are commonly used for stability training. This paper reviews and summarizes the research in the regularization and normalization for GAN. All the methods are classified into six groups: Gradient penalty, Norm normalization and regularization, Jacobian regularization, Layer normalization, Consistency regularization, and Self-supervision.

READ FULL TEXT

page 2

page 4

research
06/23/2023

Penalty Gradient Normalization for Generative Adversarial Networks

In this paper, we propose a novel normalization method called penalty gr...
research
04/13/2017

On the Effects of Batch and Weight Normalization in Generative Adversarial Networks

Generative adversarial networks (GANs) are highly effective unsupervised...
research
05/19/2020

Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies

Despite its short history, Generative Adversarial Network (GAN) has been...
research
10/13/2019

Implicit competitive regularization in GANs

Generative adversarial networks (GANs) are capable of producing high qua...
research
09/13/2019

Deep Adversarial Belief Networks

We present a novel adversarial framework for training deep belief networ...
research
07/09/2019

Mean Spectral Normalization of Deep Neural Networks for Embedded Automation

Deep Neural Networks (DNNs) have begun to thrive in the field of automat...
research
03/03/2021

Sparsity Aware Normalization for GANs

Generative adversarial networks (GANs) are known to benefit from regular...

Please sign up or login with your details

Forgot password? Click here to reset