DeepAI
Log In Sign Up

Using latent space regression to analyze and leverage compositionality in GANs

03/18/2021
by   Lucy Chai, et al.
6

In recent years, Generative Adversarial Networks have become ubiquitous in both research and public perception, but how GANs convert an unstructured latent code to a high quality output is still an open question. In this work, we investigate regression into the latent space as a probe to understand the compositional properties of GANs. We find that combining the regressor and a pretrained generator provides a strong image prior, allowing us to create composite images from a collage of random image parts at inference time while maintaining global consistency. To compare compositional properties across different generators, we measure the trade-offs between reconstruction of the unrealistic input and image quality of the regenerated samples. We find that the regression approach enables more localized editing of individual image parts compared to direct editing in the latent space, and we conduct experiments to quantify this independence effect. Our method is agnostic to the semantics of edits, and does not require labels or predefined concepts during training. Beyond image composition, our method extends to a number of related applications, such as image inpainting or example-based image editing, which we demonstrate on several GANs and datasets, and because it uses only a single forward pass, it can operate in real-time. Code is available on our project page: https://chail.github.io/latent-composition/.

READ FULL TEXT

page 16

page 20

page 23

page 24

page 27

page 28

page 29

page 30

04/30/2021

StyleMapGAN: Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing

Generative adversarial networks (GANs) synthesize realistic images from ...
11/04/2021

EditGAN: High-Precision Semantic Image Editing

Generative adversarial networks (GANs) have recently found applications ...
07/09/2021

Semantic and Geometric Unfolding of StyleGAN Latent Space

Generative adversarial networks (GANs) have proven to be surprisingly ef...
10/20/2022

Diffusion Models already have a Semantic Latent Space

Diffusion models achieve outstanding generative performance in various d...
11/17/2022

Assessing Neural Network Robustness via Adversarial Pivotal Tuning

The ability to assess the robustness of image classifiers to a diverse s...
05/31/2022

PandA: Unsupervised Learning of Parts and Appearances in the Feature Maps of GANs

Recent advances in the understanding of Generative Adversarial Networks ...
12/13/2020

Improved StyleGAN Embedding: Where are the Good Latents?

StyleGAN is able to produce photorealistic images that are almost indist...

Code Repositories

latent-mix

latent-composition and augmix


view repo