Small-GAN: Speeding Up GAN Training Using Core-sets

10/29/2019
by   Samarth Sinha, et al.
22

Recent work by Brock et al. (2018) suggests that Generative Adversarial Networks (GANs) benefit disproportionately from large mini-batch sizes. Unfortunately, using large batches is slow and expensive on conventional hardware. Thus, it would be nice if we could generate batches that were effectively large though actually small. In this work, we propose a method to do this, inspired by the use of Coreset-selection in active learning. When training a GAN, we draw a large batch of samples from the prior and then compress that batch using Coreset-selection. To create effectively large batches of 'real' images, we create a cached dataset of Inception activations of each training image, randomly project them down to a smaller dimension, and then use Coreset-selection on those projected activations at training time. We conduct experiments showing that this technique substantially reduces training time and memory usage for modern GAN variants, that it reduces the fraction of dropped modes in a synthetic dataset, and that it allows GANs to reach a new state of the art in anomaly detection.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/30/2023

Stylized Projected GAN: A Novel Architecture for Fast and Realistic Image Generation

Generative Adversarial Networks are used for generating the data using a...
research
10/13/2019

Image Generation and Recognition (Emotions)

Generative Adversarial Networks (GANs) were proposed in 2014 by Goodfell...
research
05/27/2019

Unsupervised Learning of Anomaly Detection from Contaminated Image Data using Simultaneous Encoder Training

Anomaly detection in high-dimensional data, such as images, is a challen...
research
09/05/2022

Dynamics of Fourier Modes in Torus Generative Adversarial Networks

Generative Adversarial Networks (GANs) are powerful Machine Learning mod...
research
02/28/2021

Ultra-Data-Efficient GAN Training: Drawing A Lottery Ticket First, Then Training It Toughly

Training generative adversarial networks (GANs) with limited data genera...
research
02/23/2018

Is Generator Conditioning Causally Related to GAN Performance?

Recent work (Pennington et al, 2017) suggests that controlling the entir...
research
04/18/2019

Reducing Noise in GAN Training with Variance Reduced Extragradient

Using large mini-batches when training generative adversarial networks (...

Please sign up or login with your details

Forgot password? Click here to reset