Understanding the Effectiveness of Lipschitz Constraint in Training of GANs via Gradient Analysis

07/02/2018
by   Zhiming Zhou, et al.
4

This paper aims to bring a new perspective for understanding GANs, by delving deeply into the key factors that lead to the failure and success of training of GANs. Specifically, (i) we study the value surface of the optimal discriminative function, from which we show that the fundamental source of failure in training of GANs stems from the unwarranted gradient direction; (ii) we show that Lipschitz constraint is not always necessary for evaluating Wasserstein distance, and we further demonstrate that, without Lipschitz constraint, Wasserstein GAN may also fail in the same way as other GANs; (iii) we theoretically show that Lipschitz constraint is generally a powerful tool to guarantee meaningful gradient directions and we further propose a generalized family of GAN formulations based on Lipschitz constraint, where Wasserstein GAN is a special case.

READ FULL TEXT

page 3

page 4

page 6

page 7

page 14

page 16

page 17

research
12/07/2020

Sobolev Wasserstein GAN

Wasserstein GANs (WGANs), built upon the Kantorovich-Rubinstein (KR) dua...
research
02/15/2019

Lipschitz Generative Adversarial Nets

In this paper we study the convergence of generative adversarial network...
research
02/26/2021

Moreau-Yosida f-divergences

Variational representations of f-divergences are central to many machine...
research
09/26/2017

On the regularization of Wasserstein GANs

Since their invention, generative adversarial networks (GANs) have becom...
research
11/18/2018

GAN-QP: A Novel GAN Framework without Gradient Vanishing and Lipschitz Constraint

We know SGAN may have a risk of gradient vanishing. A significant improv...
research
10/05/2018

Local Stability and Performance of Simple Gradient Penalty mu-Wasserstein GAN

Wasserstein GAN(WGAN) is a model that minimizes the Wasserstein distance...
research
12/03/2018

A Wasserstein GAN model with the total variational regularization

It is well known that the generative adversarial nets (GANs) are remarka...

Please sign up or login with your details

Forgot password? Click here to reset