3D GAN Inversion for Controllable Portrait Image Animation

03/25/2022
by   Connor Z. Lin, et al.
0

Millions of images of human faces are captured every single day; but these photographs portray the likeness of an individual with a fixed pose, expression, and appearance. Portrait image animation enables the post-capture adjustment of these attributes from a single image while maintaining a photorealistic reconstruction of the subject's likeness or identity. Still, current methods for portrait image animation are typically based on 2D warping operations or manipulations of a 2D generative adversarial network (GAN) and lack explicit mechanisms to enforce multi-view consistency. Thus these methods may significantly alter the identity of the subject, especially when the viewpoint relative to the camera is changed. In this work, we leverage newly developed 3D GANs, which allow explicit control over the pose of the image subject with multi-view consistency. We propose a supervision strategy to flexibly manipulate expressions with 3D morphable models, and we show that the proposed method also supports editing appearance attributes, such as age or hairstyle, by interpolating within the latent space of the GAN. The proposed technique for portrait image animation outperforms previous methods in terms of image quality, identity preservation, and pose transfer while also supporting attribute editing.

READ FULL TEXT

page 2

page 3

page 4

page 5

page 7

page 8

page 9

page 13

research
10/13/2022

3D GAN Inversion with Pose Optimization

With the recent advances in NeRF-based 3D aware GANs quality, projecting...
research
11/17/2022

UMFuse: Unified Multi View Fusion for Human Editing applications

The vision community has explored numerous pose guided human editing met...
research
01/07/2021

GAN-Control: Explicitly Controllable GANs

We present a framework for training GANs with explicit control over gene...
research
03/13/2021

PhotoApp: Photorealistic Appearance Editing of Head Portraits

Photorealistic editing of portraits is a challenging task as humans are ...
research
02/01/2021

Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation

Controllable semantic image editing enables a user to change entire imag...
research
09/12/2022

Explicitly Controllable 3D-Aware Portrait Generation

In contrast to the traditional avatar creation pipeline which is a costl...
research
09/20/2021

FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold

Current Generative Adversarial Networks (GANs) produce photorealistic re...

Please sign up or login with your details

Forgot password? Click here to reset