Unsupervised Discovery of Interpretable Directions in the GAN Latent Space

02/10/2020
by   Andrey Voynov, et al.
13

The latent spaces of typical GAN models often have semantically meaningful directions. Moving in these directions corresponds to human-interpretable image transformations, such as zooming or recoloring, enabling a more controllable generation process. However, the discovery of such directions is currently performed in a supervised manner, requiring human labels, pretrained models, or some form of self-supervision. These requirements can severely limit a range of directions existing approaches can discover. In this paper, we introduce an unsupervised method to identify interpretable directions in the latent space of a pretrained GAN model. By a simple model-agnostic procedure, we find directions corresponding to sensible semantic manipulations without any form of (self-)supervision. Furthermore, we reveal several non-trivial findings, which would be difficult to obtain by existing methods, e.g., a direction corresponding to background removal. As an immediate practical benefit of our work, we show how to exploit this finding to achieve a new state-of-the-art for the problem of saliency detection.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

page 8

research
04/02/2021

LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions

Recent research has shown great potential for finding interpretable dire...
research
07/20/2022

Interpreting Latent Spaces of Generative Models for Medical Images using Unsupervised Methods

Generative models such as Generative Adversarial Networks (GANs) and Var...
research
05/17/2021

Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models

Recent research has shown that numerous human-interpretable directions e...
research
12/20/2020

Latent Compass: Creation by Navigation

In Marius von Senden's Space and Sight, a newly sighted blind patient de...
research
12/09/2020

GAN Steerability without optimization

Recent research has shown remarkable success in revealing "steering" dir...
research
01/20/2023

Spatial Steerability of GANs via Self-Supervision from Discriminator

Generative models make huge progress to the photorealistic image synthes...
research
08/13/2022

Self-supervised Matting-specific Portrait Enhancement and Generation

We resolve the ill-posed alpha matting problem from a completely differe...

Please sign up or login with your details

Forgot password? Click here to reset