Synthesize-It-Classifier: Learning a Generative Classifier through RecurrentSelf-analysis

03/26/2021
by   Arghya Pal, et al.
10

In this work, we show the generative capability of an image classifier network by synthesizing high-resolution, photo-realistic, and diverse images at scale. The overall methodology, called Synthesize-It-Classifier (STIC), does not require an explicit generator network to estimate the density of the data distribution and sample images from that, but instead uses the classifier's knowledge of the boundary to perform gradient ascent w.r.t. class logits and then synthesizes images using Gram Matrix Metropolis Adjusted Langevin Algorithm (GRMALA) by drawing on a blank canvas. During training, the classifier iteratively uses these synthesized images as fake samples and re-estimates the class boundary in a recurrent fashion to improve both the classification accuracy and quality of synthetic images. The STIC shows the mixing of the hard fake samples (i.e. those synthesized by the one hot class conditioning), and the soft fake samples (which are synthesized as a convex combination of classes, i.e. a mixup of classes) improves class interpolation. We demonstrate an Attentive-STIC network that shows an iterative drawing of synthesized images on the ImageNet dataset that has thousands of classes. In addition, we introduce the synthesis using a class conditional score classifier (Score-STIC) instead of a normal image classifier and show improved results on several real-world datasets, i.e. ImageNet, LSUN, and CIFAR 10.

READ FULL TEXT

page 1

page 6

page 15

page 16

page 17

page 18

page 19

page 20

10/30/2016

Conditional Image Synthesis With Auxiliary Classifier GANs

Synthesizing high resolution photorealistic images has been a long-stand...
04/25/2017

Introspective Classifier Learning: Empower Generatively

In this paper we propose introspective classifier learning (ICL) that em...
04/01/2018

SampleAhead: Online Classifier-Sampler Communication for Learning from Synthesized Data

State-of-the-art techniques of artificial intelligence, in particular de...
11/30/2016

Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space

Generating high-resolution, photo-realistic images has been a long-stand...
04/12/2021

Adversarial Open Domain Adaption for Sketch-to-Photo Synthesis

In this paper, we explore the open-domain sketch-to-photo translation, w...
03/19/2020

Pose Augmentation: Class-agnostic Object Pose Transformation for Object Recognition

Object pose increases interclass object variance which makes object reco...
12/18/2019

Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion

We introduce DeepInversion, a new method for synthesizing images from th...

Code Repositories

STIC

Project Page of Synthesize-It-Classifier paper, CVPR 2021


view repo