Barbershop: GAN-based Image Compositing using Segmentation Masks

Seamlessly blending features from multiple images is extremely challenging because of complex relationships in lighting, geometry, and partial occlusion which cause coupling between different parts of the image. Even though recent work on GANs enables synthesis of realistic hair or faces, it remains difficult to combine them into a single, coherent, and plausible image rather than a disjointed set of image patches. We present a novel solution to image blending, particularly for the problem of hairstyle transfer, based on GAN-inversion. We propose a novel latent space for image blending which is better at preserving detail and encoding spatial information, and propose a new GAN-embedding algorithm which is able to slightly modify images to conform to a common segmentation mask. Our novel representation enables the transfer of the visual properties from multiple reference images including specific details such as moles and wrinkles, and because we do image blending in a latent-space we are able to synthesize images that are coherent. Our approach avoids blending artifacts present in other approaches and finds a globally consistent image. Our results demonstrate a significant improvement over the current state of the art in a user study, with users preferring our blending solution over 95 percent of the time.

READ FULL TEXT

page 1

page 4

page 5

page 8

page 9

page 10

page 11

03/05/2021

LOHO: Latent Optimization of Hairstyles via Orthogonalization

Hairstyle transfer is challenging due to hair structure differences in t...
12/15/2019

Image Processing Using Multi-Code GAN Prior

Despite the success of Generative Adversarial Networks (GANs) in image s...
03/14/2022

InsetGAN for Full-Body Image Generation

While GANs can produce photo-realistic images in ideal conditions for ce...
06/16/2022

Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing

Existing GAN inversion and editing methods work well for aligned objects...
08/25/2022

High-Fidelity Image Inpainting with GAN Inversion

Image inpainting seeks a semantically consistent way to recover the corr...
11/29/2018

Diverse Image Synthesis from Semantic Layouts via Conditional IMLE

Most existing methods for conditional image synthesis are only able to g...
03/06/2020

Anysize GAN: A solution to the image-warping problem

We propose a new type of General Adversarial Network (GAN) to resolve a ...

Code Repositories

Barbershop

Barbershop: GAN-based Image Compositing using Segmentation Masks


view repo