DeepAI AI Chat
Log In Sign Up

MarioNETte: Few-shot Face Reenactment Preserving Identity of Unseen Targets

by   Sungjoo Ha, et al.
Hyperconnect, Inc.

When there is a mismatch between the target identity and the driver identity, face reenactment suffers severe degradation in the quality of the result, especially in a few-shot setting. The identity preservation problem, where the model loses the detailed information of the target leading to a defective output, is the most common failure mode. The problem has several potential sources such as the identity of the driver leaking due to the identity mismatch, or dealing with unseen large poses. To overcome such problems, we introduce components that address the mentioned problem: image attention block, target feature alignment, and landmark transformer. Through attending and warping the relevant features, the proposed architecture, called MarioNETte, produces high-quality reenactments of unseen identities in a few-shot setting. In addition, the landmark transformer dramatically alleviates the identity preservation problem by isolating the expression geometry through landmark disentanglement. Comprehensive experiments are performed to verify that the proposed framework can generate highly realistic faces, outperforming all other baselines, even under a significant mismatch of facial characteristics between the target and the driver.


page 14

page 15

page 16

page 17

page 18

page 19

page 20

page 21


One-Shot Identity-Preserving Portrait Reenactment

We present a deep learning-based framework for portrait reenactment from...

LI-Net: Large-Pose Identity-Preserving Face Reenactment Network

Face reenactment is a challenging task, as it is difficult to maintain a...

FaceSwapNet: Landmark Guided Many-to-Many Face Reenactment

Recent face reenactment studies have achieved remarkable success either ...

Semantic-aware One-shot Face Re-enactment with Dense Correspondence Estimation

One-shot face re-enactment is a challenging task due to the identity mis...

ActGAN: Flexible and Efficient One-shot Face Reenactment

This paper introduces ActGAN - a novel end-to-end generative adversarial...

Protecting Celebrities with Identity Consistency Transformer

In this work we propose Identity Consistency Transformer, a novel face f...

Geometry Driven Progressive Warping for One-Shot Face Animation

Face animation aims at creating photo-realistic portrait videos with ani...