DeepAI AI Chat
Log In Sign Up

Neuromuscular Control of the Face-Head-Neck Biomechanical Complex With Learning-Based Expression Transfer From Images and Videos

by   Xiao S. Zeng, et al.

The transfer of facial expressions from people to 3D face models is a classic computer graphics problem. In this paper, we present a novel, learning-based approach to transferring facial expressions and head movements from images and videos to a biomechanical model of the face-head-neck complex. Leveraging the Facial Action Coding System (FACS) as an intermediate representation of the expression space, we train a deep neural network to take in FACS Action Units (AUs) and output suitable facial muscle and jaw activation signals for the musculoskeletal model. Through biomechanical simulation, the activations deform the facial soft tissues, thereby transferring the expression to the model. Our approach has advantages over previous approaches. First, the facial expressions are anatomically consistent as our biomechanical model emulates the relevant anatomy of the face, head, and neck. Second, by training the neural network using data generated from the biomechanical model itself, we eliminate the manual effort of data collection for expression transfer. The success of our approach is demonstrated through experiments involving the transfer onto our face-head-neck model of facial expressions and head poses from a range of facial images and videos.


page 1

page 7

page 8

page 9


What is the relationship between face alignment and facial expression recognition?

Face expression recognition is still a complex task, particularly due to...

CycleGAN Face-off

Face-off is an interesting case of style transfer where the facial expre...

Detecting Deep-Fake Videos from Appearance and Behavior

Synthetically-generated audios and videos – so-called deep fakes – conti...

VariTex: Variational Neural Face Textures

Deep generative models can synthesize photorealistic images of human fac...

Facial expressions can detect Parkinson's disease: preliminary evidence from videos collected online

One of the symptoms of Parkinson's disease (PD) is hypomimia or reduced ...

Kernel Projection of Latent Structures Regression for Facial Animation Retargeting

Inspired by kernel methods that have been used extensively in achieving ...

Real-time Simultaneous 3D Head Modeling and Facial Motion Capture with an RGB-D camera

We propose a method to build in real-time animated 3D head models using ...