Audio-Driven Emotional Video Portraits

04/15/2021
by   Xinya Ji, et al.
0

Despite previous success in generating audio-driven talking heads, most of the previous studies focus on the correlation between speech content and the mouth shape. Facial emotion, which is one of the most important features on natural human faces, is always neglected in their methods. In this work, we present Emotional Video Portraits (EVP), a system for synthesizing high-quality video portraits with vivid emotional dynamics driven by audios. Specifically, we propose the Cross-Reconstructed Emotion Disentanglement technique to decompose speech into two decoupled spaces, i.e., a duration-independent emotion space and a duration dependent content space. With the disentangled features, dynamic 2D emotional facial landmarks can be deduced. Then we propose the Target-Adaptive Face Synthesis technique to generate the final high-quality video portraits, by bridging the gap between the deduced landmarks and the natural head poses of target videos. Extensive experiments demonstrate the effectiveness of our method both qualitatively and quantitatively.

READ FULL TEXT

page 1

page 3

page 6

page 8

research
06/15/2023

Emotional Speech-Driven Animation with Content-Emotion Disentanglement

To be widely adopted, 3D facial avatars need to be animated easily, real...
research
09/10/2023

Efficient Emotional Adaptation for Audio-Driven Talking-Head Generation

Audio-driven talking-head synthesis is a popular research topic for virt...
research
04/25/2021

3D-TalkEmo: Learning to Synthesize 3D Emotional Talking Head

Impressive progress has been made in audio-driven 3D facial animation re...
research
10/02/2019

Animating Face using Disentangled Audio Representations

All previous methods for audio-driven talking head generation assume the...
research
05/30/2022

EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model

Although significant progress has been made to audio-driven talking face...
research
06/13/2023

Parametric Implicit Face Representation for Audio-Driven Facial Reenactment

Audio-driven facial reenactment is a crucial technique that has a range ...
research
05/02/2022

Emotion-Controllable Generalized Talking Face Generation

Despite the significant progress in recent years, very few of the AI-bas...

Please sign up or login with your details

Forgot password? Click here to reset