DeepAI AI Chat
Log In Sign Up

Free-HeadGAN: Neural Talking Head Synthesis with Explicit Gaze Control

by   Michail Christos Doukas, et al.

We present Free-HeadGAN, a person-generic neural talking head synthesis system. We show that modeling faces with sparse 3D facial landmarks are sufficient for achieving state-of-the-art generative performance, without relying on strong statistical priors of the face, such as 3D Morphable Models. Apart from 3D pose and facial expressions, our method is capable of fully transferring the eye gaze, from a driving actor to a source identity. Our complete pipeline consists of three components: a canonical 3D key-point estimator that regresses 3D pose and expression-related deformations, a gaze estimation network and a generator that is built upon the architecture of HeadGAN. We further experiment with an extension of our generator to accommodate few-shot learning using an attention mechanism, in case more than one source images are available. Compared to the latest models for reenactment and motion transfer, our system achieves higher photo-realism combined with superior identity preservation, while offering explicit gaze control.


page 1

page 5

page 8

page 9

page 11

page 13

page 14

page 15


Head2HeadFS: Video-based Head Reenactment with Few-shot Learning

Over the past years, a substantial amount of work has been done on the p...

Head2Head: Video-based Neural Head Synthesis

In this paper, we propose a novel machine learning architecture for faci...

HeadOn: Real-time Reenactment of Human Portrait Videos

We propose HeadOn, the first real-time source-to-target reenactment appr...

HoloFace: Augmenting Human-to-Human Interactions on HoloLens

We present HoloFace, an open-source framework for face alignment, head p...

DIF : Dataset of Intoxicated Faces for Drunk Person Identification

Traffic accidents cause over a million deaths every year, of which a lar...

HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces

In this paper, we present our method for neural face reenactment, called...

Towards a complete 3D morphable model of the human head

Three-dimensional Morphable Models (3DMMs) are powerful statistical tool...