Log In Sign Up

X2Face: A network for controlling face generation by using images, audio, and pose codes

by   Olivia Wiles, et al.

The objective of this paper is a neural network model that controls the pose and expression of a given face, using another face or modality (e.g. audio). This model can then be used for lightweight, sophisticated video and image editing. We make the following three contributions. First, we introduce a network, X2Face, that can control a source face (specified by one or more frames) using another face in a driving frame to produce a generated frame with the identity of the source frame but the pose and expression of the face in the driving frame. Second, we propose a method for training the network fully self-supervised using a large collection of video data. Third, we show that the generation process can be driven by other modalities, such as audio or pose codes, without any further training of the network. The generation results for driving a face with another face are compared to state-of-the-art self-supervised/supervised methods. We show that our approach is more robust than other methods, as it makes fewer assumptions about the input data. We also show examples of using our framework for video face editing.


page 9

page 11

page 12

page 14

page 20

page 21

page 22

page 23


ICface: Interpretable and Controllable Face Reenactment Using GANs

This paper presents a generic face animator that is able to control the ...

Self-supervised learning of a facial attribute embedding from video

We propose a self-supervised framework for learning facial attributes by...

Text-based Editing of Talking-head Video

Editing talking-head video to change the speech content or to remove fil...

Live Face De-Identification in Video

We propose a method for face de-identification that enables fully automa...

Image Animation with Perturbed Masks

We present a novel approach for image-animation of a source image by a d...

3D Face Pose and Animation Tracking via Eigen-Decomposition based Bayesian Approach

This paper presents a new method to track both the face pose and the fac...

StyleRig: Rigging StyleGAN for 3D Control over Portrait Images

StyleGAN generates photorealistic portrait images of faces with eyes, te...