Everything's Talkin': Pareidolia Face Reenactment

by   Linsen Song, et al.

We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video. For the large differences between pareidolia face reenactment and traditional human face reenactment, two main challenges are introduced, i.e., shape variance and texture variance. In this work, we propose a novel Parametric Unsupervised Reenactment Algorithm to tackle these two challenges. Specifically, we propose to decompose the reenactment into three catenate processes: shape modeling, motion transfer and texture synthesis. With the decomposition, we introduce three crucial components, i.e., Parametric Shape Modeling, Expansionary Motion Transfer and Unsupervised Texture Synthesizer, to overcome the problems brought by the remarkably variances on pareidolia faces. Extensive experiments show the superior performance of our method both qualitatively and quantitatively. Code, model and data are available on our project page.


page 1

page 3

page 5

page 6

page 7

page 8

page 12

page 13


Disentangled Lifespan Face Synthesis

A lifespan face synthesis (LFS) model aims to generate a set of photo-re...

One-shot Face Reenactment

To enable realistic shape (e.g. pose and expression) transfer, existing ...

SOGAN: 3D-Aware Shadow and Occlusion Robust GAN for Makeup Transfer

In recent years, virtual makeup applications have become more and more p...

Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection

Non-parametric face modeling aims to reconstruct 3D face only from image...

Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis

We tackle the human motion imitation, appearance transfer, and novel vie...

Co-projection-plane based 3-D padding for polyhedron projection for 360-degree video

The polyhedron projection for 360-degree video is becoming more and more...

Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture

The problem of deforming an artist-drawn caricature according to a given...