DeepAI AI Chat
Log In Sign Up

Training GANs with predictive projection centripetal acceleration

10/07/2020
by   Li Keke, et al.
0

Although remarkable successful in practice, training generative adversarial networks(GANs) is still quite difficult and iteratively prone to cyclic behaviors, as GANs need to solve a non-convex non-concave min-max game using a gradient descent ascent (GDA) method. Motivated by the ideas of simultaneous centripetal acceleration (SCA) and modified predictive methods (MPM), we propose a novel predictive projection centripetal acceleration (PPCA) methods to alleviate the cyclic behaviors. Besides, under suitable assumptions, we show that the difference between the signed vector of partial derivatives at t + 1 and t is orthogonal to the signed vector of partial derivatives at t for GDA, and the last-iterate exponential convergence on the bilinear game. Finally, numerical simulations are conducted by PPCA in GANs setting, and the results illustrate the effectiveness of our approach.

READ FULL TEXT
02/24/2019

Training GANs with Centripetal Acceleration

Training generative adversarial networks (GANs) often suffers from cycli...
04/12/2021

Understanding Overparameterization in Generative Adversarial Networks

A broad class of unsupervised deep learning methods such as Generative A...
11/10/2021

Training Generative Adversarial Networks with Adaptive Composite Gradient

The wide applications of Generative adversarial networks benefit from th...
04/22/2019

Training generative networks using random discriminators

In recent years, Generative Adversarial Networks (GANs) have drawn a lot...
07/12/2021

Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions

Generative Adversarial Networks (GANs) are commonly used for modeling co...
06/12/2018

The Unusual Effectiveness of Averaging in GAN Training

We show empirically that the optimal strategy of parameter averaging in ...