Disentangling Latent Space for Unsupervised Semantic Face Editing

11/05/2020
by   Kanglin Liu, et al.
10

Editing facial images created by StyleGAN is a popular research topic with important applications. Through editing the latent vectors, it is possible to control the facial attributes such as smile, age, etc. However, facial attributes are entangled in the latent space and this makes it very difficult to independently control a specific attribute without affecting the others. The key to developing neat semantic control is to completely disentangle the latent space and perform image editing in an unsupervised manner. In this paper, we present a new technique termed Structure-Texture Independent Architecture with Weight Decomposition and Orthogonal Regularization (STIA-WO) to disentangle the latent space. The GAN model, applying STIA-WO, is referred to as STGAN-WO. STGAN-WO performs weight decomposition by utilizing the style vector to construct a fully controllable weight matrix for controlling the image synthesis, and utilizes orthogonal regularization to ensure each entry of the style vector only controls one factor of variation. To further disentangle the facial attributes, STGAN-WO introduces a structure-texture independent architecture which utilizes two independently and identically distributed (i.i.d.) latent vectors to control the synthesis of the texture and structure components in a disentangled way.Unsupervised semantic editing is achieved by moving the latent code in the coarse layers along its orthogonal directions to change texture related attributes or changing the latent code in the fine layers to manipulate structure related ones. We present experimental results which show that our new STGAN-WO can achieve better attribute editing than state of the art methods (The code is available at https://github.com/max-liu-112/STGAN-WO)

READ FULL TEXT

page 1

page 5

page 8

page 9

research
05/26/2021

Disentangled Face Attribute Editing via Instance-Aware Latent Space Search

Recent works have shown that a rich set of semantic directions exist in ...
research
07/15/2023

Adaptive Nonlinear Latent Transformation for Conditional Face Editing

Recent works for face editing usually manipulate the latent space of Sty...
research
09/11/2023

Semantic Latent Decomposition with Normalizing Flows for Face Editing

Navigating in the latent space of StyleGAN has shown effectiveness for f...
research
08/17/2021

Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation

Unsupervised disentanglement learning is a crucial issue for understandi...
research
03/05/2021

LOHO: Latent Optimization of Hairstyles via Orthogonalization

Hairstyle transfer is challenging due to hair structure differences in t...
research
03/31/2022

TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing

Recent advances like StyleGAN have promoted the growth of controllable f...
research
02/24/2023

Unsupervised Discovery of Semantic Latent Directions in Diffusion Models

Despite the success of diffusion models (DMs), we still lack a thorough ...

Please sign up or login with your details

Forgot password? Click here to reset