DeepAI AI Chat
Log In Sign Up

Improving GAN Training with Probability Ratio Clipping and Sample Reweighting

by   Yue Wu, et al.

Despite success on a wide range of problems related to vision, generative adversarial networks (GANs) can suffer from inferior performance due to unstable training, especially for text generation. we propose a new variational GAN training framework which enjoys superior training stability. Our approach is inspired by a connection of GANs and reinforcement learning under a variational perspective. The connection leads to (1) probability ratio clipping that regularizes generator training to prevent excessively large updates, and (2) a sample re-weighting mechanism that stabilizes discriminator training by downplaying bad-quality fake samples. We provide theoretical analysis on the convergence of our approach. By plugging the training approach in diverse state-of-the-art GAN architectures, we obtain significantly improved performance over a range of tasks, including text generation, text style transfer, and image generation.


page 6

page 13

page 14


Adversarial Text Generation via Feature-Mover's Distance

Generative adversarial networks (GANs) have achieved significant success...

Improved Training of Mixture-of-Experts Language GANs

Despite the dramatic success in image generation, Generative Adversarial...

TextKD-GAN: Text Generation using KnowledgeDistillation and Generative Adversarial Networks

Text generation is of particular interest in many NLP applications such ...

ColdGANs: Taming Language GANs with Cautious Sampling Strategies

Training regimes based on Maximum Likelihood Estimation (MLE) suffer fro...

Adversarial Text Generation Without Reinforcement Learning

Generative Adversarial Networks (GANs) have experienced a recent surge i...

Generating Text through Adversarial Training using Skip-Thought Vectors

In the past few years, various advancements have been made in generative...

Diffusion-GAN: Training GANs with Diffusion

For stable training of generative adversarial networks (GANs), injecting...