Mesh Guided One-shot Face Reenactment using Graph Convolutional Networks

08/18/2020
by   Guangming Yao, et al.
10

Face reenactment aims to animate a source face image to a different pose and expression provided by a driving image. Existing approaches are either designed for a specific identity, or suffer from the identity preservation problem in the one-shot or few-shot scenarios. In this paper, we introduce a method for one-shot face reenactment, which uses the reconstructed 3D meshes (i.e., the source mesh and driving mesh) as guidance to learn the optical flow needed for the reenacted face synthesis. Technically, we explicitly exclude the driving face's identity information in the reconstructed driving mesh. In this way, our network can focus on the motion estimation for the source face without the interference of driving face shape. We propose a motion net to learn the face motion, which is an asymmetric autoencoder. The encoder is a graph convolutional network (GCN) that learns a latent motion vector from the meshes, and the decoder serves to produce an optical flow image from the latent vector with CNNs. Compared to previous methods using sparse keypoints to guide the optical flow learning, our motion net learns the optical flow directly from 3D dense meshes, which provide the detailed shape and pose information for the optical flow, so it can achieve more accurate expression and pose on the reenacted face. Extensive experiments show that our method can generate high-quality results and outperforms state-of-the-art methods in both qualitative and quantitative comparisons.

READ FULL TEXT

page 1

page 3

page 6

page 8

research
04/07/2021

Single Source One Shot Reenactment using Weighted motion From Paired Feature Points

Image reenactment is a task where the target object in the source image ...
research
05/04/2021

Self-Supervised Approach for Facial Movement Based Optical Flow

Computing optical flow is a fundamental problem in computer vision. Howe...
research
03/27/2022

Thin-Plate Spline Motion Model for Image Animation

Image animation brings life to the static object in the source image acc...
research
11/23/2022

Semantic-aware One-shot Face Re-enactment with Dense Correspondence Estimation

One-shot face re-enactment is a challenging task due to the identity mis...
research
07/02/2021

Deep Mesh Prior: Unsupervised Mesh Restoration using Graph Convolutional Networks

This paper addresses mesh restoration problems, i.e., denoising and comp...
research
05/02/2022

Emotion-Controllable Generalized Talking Face Generation

Despite the significant progress in recent years, very few of the AI-bas...
research
07/29/2022

Generating Complex 4D Expression Transitions by Learning Face Landmark Trajectories

In this work, we address the problem of 4D facial expressions generation...

Please sign up or login with your details

Forgot password? Click here to reset