Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild

05/15/2023
by   Dafei Qin, et al.
0

We propose an end-to-end deep-learning approach for automatic rigging and retargeting of 3D models of human faces in the wild. Our approach, called Neural Face Rigging (NFR), holds three key properties: (i) NFR's expression space maintains human-interpretable editing parameters for artistic controls; (ii) NFR is readily applicable to arbitrary facial meshes with different connectivity and expressions; (iii) NFR can encode and produce fine-grained details of complex expressions performed by arbitrary subjects. To the best of our knowledge, NFR is the first approach to provide realistic and controllable deformations of in-the-wild facial meshes, without the manual creation of blendshapes or correspondence. We design a deformation autoencoder and train it through a multi-dataset training scheme, which benefits from the unique advantages of two data sources: a linear 3DMM with interpretable control parameters as in FACS, and 4D captures of real faces with fine-grained details. Through various experiments, we show NFR's ability to automatically produce realistic and accurate facial deformations across a wide range of existing datasets as well as noisy facial scans in-the-wild, while providing artist-controlled, editable parameters.

READ FULL TEXT

page 1

page 6

page 7

page 10

page 11

page 12

page 13

research
07/14/2020

Personalized Face Modeling for Improved Face Reconstruction and Motion Retargeting

Traditional methods for image-based 3D face reconstruction and facial mo...
research
03/26/2019

Photo-Realistic Facial Details Synthesis from Single Immage

We present a single-image 3D face synthesis technique that can handle ch...
research
12/07/2020

Learning an Animatable Detailed 3D Face Model from In-The-Wild Images

While current monocular 3D face reconstruction methods can recover fine ...
research
05/12/2023

BlendFields: Few-Shot Example-Driven Facial Modeling

Generating faithful visualizations of human faces requires capturing bot...
research
05/27/2022

Video2StyleGAN: Disentangling Local and Global Variations in a Video

Image editing using a pretrained StyleGAN generator has emerged as a pow...
research
10/12/2020

Fat Pad Cages for Facial Posing

We introduce Fat Pad cages for posing facial meshes. It combines cage re...
research
03/25/2023

3D Facial Imperfection Regeneration: Deep learning approach and 3D printing prototypes

This study explores the potential of a fully convolutional mesh autoenco...

Please sign up or login with your details

Forgot password? Click here to reset