Everything's Talkin': Pareidolia Face Reenactment

04/07/2021
by   Linsen Song, et al.
0

We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video. For the large differences between pareidolia face reenactment and traditional human face reenactment, two main challenges are introduced, i.e., shape variance and texture variance. In this work, we propose a novel Parametric Unsupervised Reenactment Algorithm to tackle these two challenges. Specifically, we propose to decompose the reenactment into three catenate processes: shape modeling, motion transfer and texture synthesis. With the decomposition, we introduce three crucial components, i.e., Parametric Shape Modeling, Expansionary Motion Transfer and Unsupervised Texture Synthesizer, to overcome the problems brought by the remarkably variances on pareidolia faces. Extensive experiments show the superior performance of our method both qualitatively and quantitatively. Code, model and data are available on our project page.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

page 8

page 12

page 13

08/05/2021

Disentangled Lifespan Face Synthesis

A lifespan face synthesis (LFS) model aims to generate a set of photo-re...
08/05/2019

One-shot Face Reenactment

To enable realistic shape (e.g. pose and expression) transfer, existing ...
04/21/2021

SOGAN: 3D-Aware Shadow and Occlusion Robust GAN for Makeup Transfer

In recent years, virtual makeup applications have become more and more p...
06/15/2021

Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection

Non-parametric face modeling aims to reconstruct 3D face only from image...
09/26/2019

Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis

We tackle the human motion imitation, appearance transfer, and novel vie...
04/27/2017

Co-projection-plane based 3-D padding for polyhedron projection for 360-degree video

The polyhedron projection for 360-degree video is becoming more and more...
08/13/2020

Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture

The problem of deforming an artist-drawn caricature according to a given...