DeepAI AI Chat
Log In Sign Up

Training GANs with predictive projection centripetal acceleration

by   Li Keke, et al.

Although remarkable successful in practice, training generative adversarial networks(GANs) is still quite difficult and iteratively prone to cyclic behaviors, as GANs need to solve a non-convex non-concave min-max game using a gradient descent ascent (GDA) method. Motivated by the ideas of simultaneous centripetal acceleration (SCA) and modified predictive methods (MPM), we propose a novel predictive projection centripetal acceleration (PPCA) methods to alleviate the cyclic behaviors. Besides, under suitable assumptions, we show that the difference between the signed vector of partial derivatives at t + 1 and t is orthogonal to the signed vector of partial derivatives at t for GDA, and the last-iterate exponential convergence on the bilinear game. Finally, numerical simulations are conducted by PPCA in GANs setting, and the results illustrate the effectiveness of our approach.


Training GANs with Centripetal Acceleration

Training generative adversarial networks (GANs) often suffers from cycli...

Understanding Overparameterization in Generative Adversarial Networks

A broad class of unsupervised deep learning methods such as Generative A...

Training Generative Adversarial Networks with Adaptive Composite Gradient

The wide applications of Generative adversarial networks benefit from th...

Training generative networks using random discriminators

In recent years, Generative Adversarial Networks (GANs) have drawn a lot...

Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions

Generative Adversarial Networks (GANs) are commonly used for modeling co...

The Unusual Effectiveness of Averaging in GAN Training

We show empirically that the optimal strategy of parameter averaging in ...