DeepAI AI Chat
Log In Sign Up

Expressive Speech-driven Facial Animation with controllable emotions

by   Yutong Chen, et al.

It is in high demand to generate facial animation with high realism, but it remains a challenging task. Existing approaches of speech-driven facial animation can produce satisfactory mouth movement and lip synchronization, but show weakness in dramatic emotional expressions and flexibility in emotion control. This paper presents a novel deep learning-based approach for expressive facial animation generation from speech that can exhibit wide-spectrum facial expressions with controllable emotion type and intensity. We propose an emotion controller module to learn the relationship between the emotion variations (e.g., types and intensity) and the corresponding facial expression parameters. It enables emotion-controllable facial animation, where the target expression can be continuously adjusted as desired. The qualitative and quantitative evaluations show that the animation generated by our method is rich in facial emotional expressiveness while retaining accurate lip movement, outperforming other state-of-the-art methods.


page 3

page 4

page 5


ExprGAN: Facial Expression Editing with Controllable Expression Intensity

Facial expression editing is a challenging task as it needs a high-level...

Emotion Dependent Facial Animation from Affective Speech

In human-to-computer interaction, facial animation in synchrony with aff...

Continuously Controllable Facial Expression Editing in Talking Face Videos

Recently audio-driven talking face video generation has attracted consid...

3D-TalkEmo: Learning to Synthesize 3D Emotional Talking Head

Impressive progress has been made in audio-driven 3D facial animation re...

RARITYNet: Rarity Guided Affective Emotion Learning Framework

Inspired from the assets of handcrafted and deep learning approaches, we...

EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance

Although current neural text-to-speech (TTS) models are able to generate...

EMOCA: Emotion Driven Monocular Face Capture and Animation

As 3D facial avatars become more widely used for communication, it is cr...