DeepAI
Log In Sign Up

GANs Can Play Lottery Tickets Too

05/31/2021
by   Xuxi Chen, et al.
0

Deep generative adversarial networks (GANs) have gained growing popularity in numerous scenarios, while usually suffer from high parameter complexities for resource-constrained real-world applications. However, the compression of GANs has less been explored. A few works show that heuristically applying compression techniques normally leads to unsatisfactory results, due to the notorious training instability of GANs. In parallel, the lottery ticket hypothesis shows prevailing success on discriminative models, in locating sparse matching subnetworks capable of training in isolation to full model performance. In this work, we for the first time study the existence of such trainable matching subnetworks in deep GANs. For a range of GANs, we certainly find matching subnetworks at 67 pruning discriminator has a minor effect on the existence and quality of matching subnetworks, while the initialization weights used in the discriminator play a significant role. We then show the powerful transferability of these subnetworks to unseen tasks. Furthermore, extensive experimental results demonstrate that our found subnetworks substantially outperform previous state-of-the-art GAN compression approaches in both image generation (e.g. SNGAN) and image-to-image translation GANs (e.g. CycleGAN). Codes available at https://github.com/VITA-Group/GAN-LTH.

READ FULL TEXT

page 5

page 14

page 15

08/25/2020

GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework

Generative adversarial networks (GANs) have gained increasing popularity...
06/04/2019

Generative Adversarial Networks: A Survey and Taxonomy

Generative adversarial networks (GANs) have been extensively studied in ...
12/13/2021

DGL-GAN: Discriminator Guided Learning for GAN Compression

Generative Adversarial Networks (GANs) with high computation costs, e.g....
04/06/2021

Content-Aware GAN Compression

Generative adversarial networks (GANs), e.g., StyleGAN2, play a vital ro...
10/27/2021

Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme

Recently, a series of algorithms have been explored for GAN compression,...
02/24/2020

When Relation Networks meet GANs: Relation GANs with Triplet Loss

Though recent research has achieved remarkable progress in generating re...
01/25/2020

On the Role of Receptive Field in Unsupervised Sim-to-Real Image Translation

Generative Adversarial Networks (GANs) are now widely used for photo-rea...