HeadOn: Real-time Reenactment of Human Portrait Videos

05/29/2018
by   Justus Thies, et al.
0

We propose HeadOn, the first real-time source-to-target reenactment approach for complete human portrait videos that enables transfer of torso and head motion, face expression, and eye gaze. Given a short RGB-D video of the target actor, we automatically construct a personalized geometry proxy that embeds a parametric head, eye, and kinematic torso model. A novel real-time reenactment algorithm employs this proxy to photo-realistically map the captured motion from the source actor to the target actor. On top of the coarse geometric proxy, we propose a video-based rendering technique that composites the modified target portrait video via view- and pose-dependent texturing, and creates photo-realistic imagery of the target actor under novel torso and head poses, facial expressions, and gaze directions. To this end, we propose a robust tracking of the face and torso of the source actor. We extensively evaluate our approach and show significant improvements in enabling much greater flexibility in creating realistic reenacted output videos.

READ FULL TEXT

page 1

page 3

page 6

page 8

page 9

page 10

page 11

page 12

research
05/29/2018

Deep Video Portraits

We present a novel approach that enables photo-realistic re-animation of...
research
10/11/2016

FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality

We introduce FaceVR, a novel method for gaze-aware facial reenactment in...
research
07/29/2020

Face2Face: Real-time Face Capture and Reenactment of RGB Videos

We present Face2Face, a novel approach for real-time facial reenactment ...
research
07/24/2019

Data-Driven Physical Face Inversion

Facial animation is one of the most challenging problems in computer gra...
research
05/22/2020

Head2Head: Video-based Neural Head Synthesis

In this paper, we propose a novel machine learning architecture for faci...
research
02/08/2016

Automatic Face Reenactment

We propose an image-based, facial reenactment system that replaces the f...
research
08/03/2022

Free-HeadGAN: Neural Talking Head Synthesis with Explicit Gaze Control

We present Free-HeadGAN, a person-generic neural talking head synthesis ...

Please sign up or login with your details

Forgot password? Click here to reset