GAN Steerability without optimization

12/09/2020
by   Nurit Spingarn-Eliezer, et al.
0

Recent research has shown remarkable success in revealing "steering" directions in the latent spaces of pre-trained GANs. These directions correspond to semantically meaningful image transformations e.g., shift, zoom, color manipulations), and have similar interpretable effects across all categories that the GAN can generate. Some methods focus on user-specified transformations, while others discover transformations in an unsupervised manner. However, all existing techniques rely on an optimization procedure to expose those directions, and offer no control over the degree of allowed interaction between different transformations. In this paper, we show that "steering" trajectories can be computed in closed form directly from the generator's weights without any form of training or optimization. This applies to user-prescribed geometric transformations, as well as to unsupervised discovery of more complex effects. Our approach allows determining both linear and nonlinear trajectories, and has many advantages over previous methods. In particular, we can control whether one transformation is allowed to come on the expense of another (e.g. zoom-in with or without allowing translation to keep the object centered). Moreover, we can determine the natural end-point of the trajectory, which corresponds to the largest extent to which a transformation can be applied without incurring degradation. Finally, we show how transferring attributes between images can be achieved without optimization, even across different categories.

READ FULL TEXT

page 19

page 20

page 21

page 22

page 23

page 24

page 40

page 42

research
02/10/2020

Unsupervised Discovery of Interpretable Directions in the GAN Latent Space

The latent spaces of typical GAN models often have semantically meaningf...
research
07/20/2022

Interpreting Latent Spaces of Generative Models for Medical Images using Unsupervised Methods

Generative models such as Generative Adversarial Networks (GANs) and Var...
research
12/13/2021

Exploring Latent Dimensions of Crowd-sourced Creativity

Recently, the discovery of interpretable directions in the latent spaces...
research
10/08/2021

Toward a Visual Concept Vocabulary for GAN Latent Space

A large body of recent work has identified transformations in the latent...
research
01/15/2021

The Geometry of Deep Generative Image Models and its Applications

Generative adversarial networks (GANs) have emerged as a powerful unsupe...
research
06/22/2021

Learning Identity-Preserving Transformations on Data Manifolds

Many machine learning techniques incorporate identity-preserving transfo...

Please sign up or login with your details

Forgot password? Click here to reset