One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing

11/30/2020
by   Ting-Chun Wang, et al.
0

We propose a neural talking-head video synthesis model and demonstrate its application to video conferencing. Our model learns to synthesize a talking-head video using a source image containing the target person's appearance and a driving video that dictates the motion in the output. Our motion is encoded based on a novel keypoint representation, where the identity-specific and motion-related information is decomposed unsupervisedly. Extensive experimental validation shows that our model outperforms competing methods on benchmark datasets. Moreover, our compact keypoint representation enables a video conferencing system that achieves the same visual quality as the commercial H.264 standard while only using one-tenth of the bandwidth. Besides, we show our keypoint representation allows the user to rotate the head during synthesis, which is useful for simulating a face-to-face video conferencing experience.

READ FULL TEXT
research
10/07/2022

A Keypoint Based Enhancement Method for Audio Driven Free View Talking Head Synthesis

Audio driven talking head synthesis is a challenging task that attracts ...
research
07/15/2022

MegaPortraits: One-shot Megapixel Neural Head Avatars

In this work, we advance the neural head avatar technology to the megapi...
research
12/20/2021

Image Animation with Keypoint Mask

Motion transfer is the task of synthesizing future video frames of a sin...
research
04/27/2023

Controllable One-Shot Face Video Synthesis With Semantic Aware Prior

The one-shot talking-head synthesis task aims to animate a source image ...
research
04/24/2020

Neural Head Reenactment with Latent Pose Descriptors

We propose a neural head reenactment system, which is driven by a latent...
research
10/02/2022

Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis

In this paper, we propose a novel dual-branch Transformation-Synthesis n...
research
05/05/2023

Avatar Fingerprinting for Authorized Use of Synthetic Talking-Head Videos

Modern generators render talking-head videos with impressive levels of p...

Please sign up or login with your details

Forgot password? Click here to reset