GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework

by   Haotao Wang, et al.

Generative adversarial networks (GANs) have gained increasing popularity in various computer vision applications, and recently start to be deployed to resource-constrained mobile devices. Similar to other deep models, state-of-the-art GANs suffer from high parameter complexities. That has recently motivated the exploration of compressing GANs (usually generators). Compared to the vast literature and prevailing success in compressing deep classifiers, the study of GAN compression remains in its infancy, so far leveraging individual compression techniques instead of more sophisticated combinations. We observe that due to the notorious instability of training GANs, heuristically stacking different compression techniques will result in unsatisfactory results. To this end, we propose the first unified optimization framework combining multiple compression means for GAN compression, dubbed GAN Slimming (GS). GS seamlessly integrates three mainstream compression techniques: model distillation, channel pruning and quantization, together with the GAN minimax objective, into one unified optimization form, that can be efficiently optimized from end to end. Without bells and whistles, GS largely outperforms existing options in compressing image-to-image translation GANs. Specifically, we apply GS to compress CartoonGAN, a state-of-the-art style transfer network, by up to 47 times, with minimal visual quality degradation. Codes and pre-trained models can be found at


page 2

page 11

page 14

page 19


AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks

The compression of Generative Adversarial Networks (GANs) has lately dra...

GANs Can Play Lottery Tickets Too

Deep generative adversarial networks (GANs) have gained growing populari...

A Survey on GAN Acceleration Using Memory Compression Technique

Since its invention, Generative adversarial networks (GANs) have shown o...

DGL-GAN: Discriminator Guided Learning for GAN Compression

Generative Adversarial Networks (GANs) with high computation costs, e.g....

Self-Supervised GAN Compression

Deep learning's success has led to larger and larger models to handle mo...

Learning Efficient GANs via Differentiable Masks and co-Attention Distillation

Generative Adversarial Networks (GANs) have been widely-used in image tr...

Blur, Noise, and Compression Robust Generative Adversarial Networks

Recently, generative adversarial networks (GANs), which learn data distr...

Code Repositories


[ICML2020] "AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks" by Yonggan Fu, Wuyang Chen, Haotao Wang, Haoran Li, Yingyan Lin, Zhangyang Wang

view repo