Causal Representation Learning for Fine-Grained Face Transfer

10/04/2021
by   Gege Gao, et al.
0

Identity transfer often faces the challenge of generalizing to new situations where large pose and expression or background gaps exist between source and target face images. To improve generalization in such situations, biases take a key role <cit.>. This paper proposes an Errors-in-Variables Adapter (EVA) model to induce learning of proper generalizations by explicitly employing biases to identity estimation based on prior knowledge about the target situation. To better match the source face with the target situation in terms of pose, expression, and background factors, we model the bias as a causal effect of the target situation on source identity and estimate this effect through a controlled intervention trial. To achieve smoother transfer for the target face across the identity gap, we eliminate the target face specificity through multiple kernel regressions. The kernels are used to constrain the regressions to operate only on identity information in the internal representations of the target image, while leaving other perceptual information invariant. Combining these post-regression representations with the biased estimation for identity, EVA shows impressive performance even in the presence of large gaps, providing empirical evidence supporting the utility of the inductive biases in identity estimation.

READ FULL TEXT

page 2

page 8

research
05/31/2017

Representation Learning by Rotating Your Faces

The large pose discrepancy between two face images is one of the fundame...
research
12/01/2021

FaceTuneGAN: Face Autoencoder for Convolutional Expression Transfer Using Neural Generative Adversarial Networks

In this paper, we present FaceTuneGAN, a new 3D face model representatio...
research
04/07/2021

LI-Net: Large-Pose Identity-Preserving Face Reenactment Network

Face reenactment is a challenging task, as it is difficult to maintain a...
research
11/29/2016

Fast Face-swap Using Convolutional Neural Networks

We consider the problem of face swapping in images, where an input ident...
research
01/10/2023

Robust Human Identity Anonymization using Pose Estimation

Many outdoor autonomous mobile platforms require more human identity ano...
research
06/08/2023

ReliableSwap: Boosting General Face Swapping Via Reliable Supervision

Almost all advanced face swapping approaches use reconstruction as the p...

Please sign up or login with your details

Forgot password? Click here to reset