Cross-Domain and Disentangled Face Manipulation with 3D Guidance

04/22/2021
by   Can Wang, et al.
16

Face image manipulation via three-dimensional guidance has been widely applied in various interactive scenarios due to its semantically-meaningful understanding and user-friendly controllability. However, existing 3D-morphable-model-based manipulation methods are not directly applicable to out-of-domain faces, such as non-photorealistic paintings, cartoon portraits, or even animals, mainly due to the formidable difficulties in building the model for each specific face domain. To overcome this challenge, we propose, as far as we know, the first method to manipulate faces in arbitrary domains using human 3DMM. This is achieved through two major steps: 1) disentangled mapping from 3DMM parameters to the latent space embedding of a pre-trained StyleGAN2 that guarantees disentangled and precise controls for each semantic attribute; and 2) cross-domain adaptation that bridges domain discrepancies and makes human 3DMM applicable to out-of-domain faces by enforcing a consistent latent space embedding. Experiments and comparisons demonstrate the superiority of our high-quality semantic manipulation method on a variety of face domains with all major 3D facial attributes controllable: pose, expression, shape, albedo, and illumination. Moreover, we develop an intuitive editing interface to support user-friendly control and instant feedback. Our project page is https://cassiepython.github.io/sigasia/cddfm3d.html.

READ FULL TEXT

page 1

page 4

page 7

page 8

page 10

page 11

page 12

page 13

research
03/24/2022

IA-FaceS: A Bidirectional Method for Semantic Face Editing

Semantic face editing has achieved substantial progress in recent years....
research
05/18/2020

InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs

Although Generative Adversarial Networks (GANs) have made significant pr...
research
01/28/2019

Cross-Domain Image Manipulation by Demonstration

In this work we propose a model that can manipulate individual visual at...
research
11/09/2020

Learning to Infer Semantic Parameters for 3D Shape Editing

Many applications in 3D shape design and augmentation require the abilit...
research
08/05/2022

Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment

For visual manipulation tasks, we aim to represent image content with se...
research
09/17/2021

PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering

Generating portrait images by controlling the motions of existing faces ...
research
08/06/2020

StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows

High-quality, diverse, and photorealistic images can now be generated by...

Please sign up or login with your details

Forgot password? Click here to reset