Swapping Autoencoder for Deep Image Manipulation

07/01/2020
by   Taesung Park, et al.
2

Deep generative models have become increasingly effective at producing realistic images from randomly sampled seeds, but using such models for controllable manipulation of existing images remains challenging. We propose the Swapping Autoencoder, a deep model designed specifically for image manipulation, rather than random sampling. The key idea is to encode an image with two independent components and enforce that any swapped combination maps to a realistic image. In particular, we encourage the components to represent structure and texture, by enforcing one component to encode co-occurrent patch statistics across different parts of an image. As our method is trained with an encoder, finding the latent codes for a new input image becomes trivial, rather than cumbersome. As a result, it can be used to manipulate real input images in various ways, including texture swapping, local and global editing, and latent code vector arithmetic. Experiments on multiple datasets show that our model produces better results and is substantially more efficient compared to recent generative models.

READ FULL TEXT

page 5

page 8

page 9

page 16

page 17

page 18

page 19

page 20

research
02/12/2022

Text and Image Guided 3D Avatar Generation and Manipulation

The manipulation of latent space has recently become an interesting topi...
research
11/30/2021

EdiBERT, a generative model for image editing

Advances in computer vision are pushing the limits of im-age manipulatio...
research
11/21/2022

Exploring the Effectiveness of Mask-Guided Feature Modulation as a Mechanism for Localized Style Editing of Real Images

The success of Deep Generative Models at high-resolution image generatio...
research
03/29/2022

Proactive Image Manipulation Detection

Image manipulation detection algorithms are often trained to discriminat...
research
09/04/2023

SMPLitex: A Generative Model and Dataset for 3D Human Texture Estimation from Single Image

We propose SMPLitex, a method for estimating and manipulating the comple...
research
02/05/2023

Divide and Compose with Score Based Generative Models

While score based generative models, or diffusion models, have found suc...
research
07/04/2021

Controllable cardiac synthesis via disentangled anatomy arithmetic

Acquiring annotated data at scale with rare diseases or conditions remai...

Please sign up or login with your details

Forgot password? Click here to reset