Generalized Adversarially Learned Inference

06/15/2020
by   Yatin Dandi, et al.
0

Allowing effective inference of latent vectors while training GANs can greatly increase their applicability in various downstream tasks. Recent approaches, such as ALI and BiGAN frameworks, develop methods of inference of latent variables in GANs by adversarially training an image generator along with an encoder to match two joint distributions of image and latent vector pairs. We generalize these approaches to incorporate multiple layers of feedback on reconstructions, self-supervision, and other forms of supervision based on prior or learned knowledge about the desired solutions. We achieve this by modifying the discriminator's objective to correctly identify more than two joint distributions of tuples of an arbitrary number of random variables consisting of images, latent vectors, and other variables generated through auxiliary tasks, such as reconstruction and inpainting or as outputs of suitable pre-trained models. We design a non-saturating maximization objective for the generator-encoder pair and prove that the resulting adversarial game corresponds to a global optimum that simultaneously matches all the distributions. Within our proposed framework, we introduce a novel set of techniques for providing self-supervised feedback to the model based on properties, such as patch-level correspondence and cycle consistency of reconstructions. Through comprehensive experiments, we demonstrate the efficacy, scalability, and flexibility of the proposed approach for a variety of tasks.

READ FULL TEXT

page 7

page 8

page 16

research
06/02/2016

Adversarially Learned Inference

We introduce the adversarially learned inference (ALI) model, which join...
research
10/11/2018

Pairwise Augmented GANs with Adversarial Reconstruction Loss

We propose a novel autoencoding model called Pairwise Augmented GANs. We...
research
10/19/2020

LT-GAN: Self-Supervised GAN with Latent Transformation Detection

Generative Adversarial Networks (GANs) coupled with self-supervised task...
research
04/07/2017

It Takes (Only) Two: Adversarial Generator-Encoder Networks

We present a new autoencoder-type architecture that is trainable in an u...
research
07/17/2018

IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis

We present a novel introspective variational autoencoder (IntroVAE) mode...
research
11/25/2020

CR-Fill: Generative Image Inpainting with Auxiliary Contexutal Reconstruction

Recent deep generative inpainting methods use attention layers to allow ...

Please sign up or login with your details

Forgot password? Click here to reset