Generating Complex 4D Expression Transitions by Learning Face Landmark Trajectories

07/29/2022
by   Naima Otberdout, et al.
3

In this work, we address the problem of 4D facial expressions generation. This is usually addressed by animating a neutral 3D face to reach an expression peak, and then get back to the neutral state. In the real world though, people show more complex expressions, and switch from one expression to another. We thus propose a new model that generates transitions between different expressions, and synthesizes long and composed 4D expressions. This involves three sub-problems: (i) modeling the temporal dynamics of expressions, (ii) learning transitions between them, and (iii) deforming a generic mesh. We propose to encode the temporal evolution of expressions using the motion of a set of 3D landmarks, that we learn to generate by training a manifold-valued GAN (Motion3DGAN). To allow the generation of composed expressions, this model accepts two labels encoding the starting and the ending expressions. The final sequence of meshes is generated by a Sparse2Dense mesh Decoder (S2D-Dec) that maps the landmark displacements to a dense, per-vertex displacement of a known mesh topology. By explicitly working with motion trajectories, the model is totally independent from the identity. Extensive experiments on five public datasets show that our proposed approach brings significant improvements with respect to previous solutions, while retaining good generalization to unseen data.

READ FULL TEXT

page 1

page 5

page 7

page 10

page 12

page 13

page 14

research
05/16/2021

3D to 4D Facial Expressions Generation Guided by Landmarks

While deep learning-based 3D face generation has made a progress recentl...
research
07/23/2019

Dynamic Facial Expression Generation on Hilbert Hypersphere with Conditional Wasserstein Generative Adversarial Nets

In this work, we propose a novel approach for generating videos of the s...
research
02/06/2018

Every Smile is Unique: Landmark-Guided Diverse Smile Generation

Each smile is unique: one person surely smiles in different ways (e.g., ...
research
03/29/2023

4D Facial Expression Diffusion Model

Facial expression generation is one of the most challenging and long-sou...
research
07/19/2020

Learning to Generate Customized Dynamic 3D Facial Expressions

Recent advances in deep learning have significantly pushed the state-of-...
research
08/18/2020

Mesh Guided One-shot Face Reenactment using Graph Convolutional Networks

Face reenactment aims to animate a source face image to a different pose...
research
10/12/2020

Fat Pad Cages for Facial Posing

We introduce Fat Pad cages for posing facial meshes. It combines cage re...

Please sign up or login with your details

Forgot password? Click here to reset