DGL-GAN: Discriminator Guided Learning for GAN Compression

12/13/2021
by   Yuesong Tian, et al.
0

Generative Adversarial Networks (GANs) with high computation costs, e.g., BigGAN and StyleGAN2, have achieved remarkable results in synthesizing high resolution and diverse images with high fidelity from random noises. Reducing the computation cost of GANs while keeping generating photo-realistic images is an urgent and challenging field for their broad applications on computational resource-limited devices. In this work, we propose a novel yet simple Discriminator Guided Learning approach for compressing vanilla GAN, dubbed DGL-GAN. Motivated by the phenomenon that the teacher discriminator may contain some meaningful information, we transfer the knowledge merely from the teacher discriminator via the adversarial function. We show DGL-GAN is valid since empirically, learning from the teacher discriminator could facilitate the performance of student GANs, verified by extensive experimental findings. Furthermore, we propose a two-stage training strategy for training DGL-GAN, which can largely stabilize its training process and achieve superior performance when we apply DGL-GAN to compress the two most representative large-scale vanilla GANs, i.e., StyleGAN2 and BigGAN. Experiments show that DGL-GAN achieves state-of-the-art (SOTA) results on both StyleGAN2 (FID 2.92 on FFHQ with nearly 1/3 parameters of StyleGAN2) and BigGAN (IS 93.29 and FID 9.92 on ImageNet with nearly 1/4 parameters of BigGAN) and also outperforms several existing vanilla GAN compression techniques. Moreover, DGL-GAN is also effective in boosting the performance of original uncompressed GANs, original uncompressed StyleGAN2 boosted with DGL-GAN achieves FID 2.65 on FFHQ, which achieves a new state-of-the-art performance. Code and models are available at <https://github.com/yuesongtian/DGL-GAN>.

READ FULL TEXT

page 2

page 5

page 10

page 12

page 16

page 18

page 19

page 20

research
12/29/2022

Discriminator-Cooperated Feature Map Distillation for GAN Compression

Despite excellent performance in image generation, Generative Adversaria...
research
01/05/2018

Semantic-aware Grad-GAN for Virtual-to-Real Urban Scene Adaption

Recent advances in vision tasks (e.g., segmentation) highly depend on th...
research
01/12/2021

Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis

Training Generative Adversarial Networks (GAN) on high-fidelity images u...
research
08/16/2021

Online Multi-Granularity Distillation for GAN Compression

Generative Adversarial Networks (GANs) have witnessed prevailing success...
research
05/31/2021

GANs Can Play Lottery Tickets Too

Deep generative adversarial networks (GANs) have gained growing populari...
research
08/25/2020

GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework

Generative adversarial networks (GANs) have gained increasing popularity...
research
02/28/2021

Ultra-Data-Efficient GAN Training: Drawing A Lottery Ticket First, Then Training It Toughly

Training generative adversarial networks (GANs) with limited data genera...

Please sign up or login with your details

Forgot password? Click here to reset