Rényi Generative Adversarial Networks

06/03/2020
by   Himesh Bhatia, et al.
8

We propose a loss function for generative adversarial networks (GANs) using Rényi information measures with parameter α. More specifically, we formulate GAN's generator loss function in terms of Rényi cross-entropy functionals. We demonstrate that for any α, this generalized loss function preserves the equilibrium point satisfied by the original GAN loss based on the Jensen-Renyi divergence, a natural extension of the Jensen-Shannon divergence. We also prove that the Rényi-centric loss function reduces to the original GAN loss function as α→ 1. We show empirically that the proposed loss function, when implemented on both DCGAN (with L_1 normalization) and StyleGAN architectures, confers performance benefits by virtue of the extra degree of freedom provided by the parameter α. More specifically, we show improvements with regard to: (a) the quality of the generated images as measured via the Fréchet Inception Distance (FID) score (e.g., best FID=8.33 for RenyiStyleGAN vs 9.7 for StyleGAN when evaluated over 64×64 CelebA images) and (b) training stability. While it was applied to GANs in this study, the proposed approach is generic and can be used in other applications of information theory to deep learning, e.g., AI bias or privacy.

READ FULL TEXT

page 24

page 25

research
11/13/2016

Least Squares Generative Adversarial Networks

Unsupervised learning with generative adversarial networks (GANs) has pr...
research
06/11/2020

Cumulant GAN

Despite the continuous improvements of Generative Adversarial Networks (...
research
08/14/2023

A Unifying Generator Loss Function for Generative Adversarial Networks

A unifying α-parametrized generator loss function is introduced for a du...
research
10/20/2019

Learning GANs and Ensembles Using Discrepancy

Generative adversarial networks (GANs) generate data based on minimizing...
research
11/20/2020

Complexity Controlled Generative Adversarial Networks

One of the issues faced in training Generative Adversarial Nets (GANs) a...
research
10/16/2019

Label-Conditioned Next-Frame Video Generation with Neural Flows

Recent state-of-the-art video generation systems employ Generative Adver...
research
04/14/2022

Ensuring accurate stain reproduction in deep generative networks for virtual immunohistochemistry

Immunohistochemistry is a valuable diagnostic tool for cancer pathology....

Please sign up or login with your details

Forgot password? Click here to reset