Style Transfer for 2D Talking Head Animation

03/17/2023
by   Trong-Thang Pham, et al.
0

Audio-driven talking head animation is a challenging research topic with many real-world applications. Recent works have focused on creating photo-realistic 2D animation, while learning different talking or singing styles remains an open problem. In this paper, we present a new method to generate talking head animation with learnable style references. Given a set of style reference frames, our framework can reconstruct 2D talking head animation based on a single input image and an audio stream. Our method first produces facial landmarks motion from the audio stream and constructs the intermediate style patterns from the style reference images. We then feed both outputs into a style-aware image generator to generate the photo-realistic and fidelity 2D animation. In practice, our framework can extract the style information of a specific character and transfer it to any new static image for talking head animation. The intensive experimental results show that our method achieves better results than recent state-of-the-art approaches qualitatively and quantitatively.

READ FULL TEXT

page 1

page 3

page 8

page 9

page 10

page 11

research
09/05/2018

Neural Comic Style Transfer: Case Study

The work by Gatys et al. [1] recently showed a neural style algorithm th...
research
08/10/2021

Domain-Aware Universal Style Transfer

Style transfer aims to reproduce content images with the styles from ref...
research
10/31/2017

Audio style transfer

"Style transfer" among images has recently emerged as a very active rese...
research
01/04/2018

Neural Style Transfer for Audio Spectograms

There has been fascinating work on creating artistic transformations of ...
research
07/16/2020

Talking-head Generation with Rhythmic Head Motion

When people deliver a speech, they naturally move heads, and this rhythm...
research
06/16/2023

Unsupervised Learning of Style-Aware Facial Animation from Real Acting Performances

This paper presents a novel approach for text/speech-driven animation of...
research
07/18/2023

OPHAvatars: One-shot Photo-realistic Head Avatars

We propose a method for synthesizing photo-realistic digital avatars fro...

Please sign up or login with your details

Forgot password? Click here to reset