DeepAI
Log In Sign Up

X2Face: A network for controlling face generation by using images, audio, and pose codes

07/27/2018
by   Olivia Wiles, et al.
4

The objective of this paper is a neural network model that controls the pose and expression of a given face, using another face or modality (e.g. audio). This model can then be used for lightweight, sophisticated video and image editing. We make the following three contributions. First, we introduce a network, X2Face, that can control a source face (specified by one or more frames) using another face in a driving frame to produce a generated frame with the identity of the source frame but the pose and expression of the face in the driving frame. Second, we propose a method for training the network fully self-supervised using a large collection of video data. Third, we show that the generation process can be driven by other modalities, such as audio or pose codes, without any further training of the network. The generation results for driving a face with another face are compared to state-of-the-art self-supervised/supervised methods. We show that our approach is more robust than other methods, as it makes fewer assumptions about the input data. We also show examples of using our framework for video face editing.

READ FULL TEXT

page 9

page 11

page 12

page 14

page 20

page 21

page 22

page 23

04/03/2019

ICface: Interpretable and Controllable Face Reenactment Using GANs

This paper presents a generic face animator that is able to control the ...
08/21/2018

Self-supervised learning of a facial attribute embedding from video

We propose a self-supervised framework for learning facial attributes by...
06/04/2019

Text-based Editing of Talking-head Video

Editing talking-head video to change the speech content or to remove fil...
11/19/2019

Live Face De-Identification in Video

We propose a method for face de-identification that enables fully automa...
11/13/2020

Image Animation with Perturbed Masks

We present a novel approach for image-animation of a source image by a d...
08/29/2019

3D Face Pose and Animation Tracking via Eigen-Decomposition based Bayesian Approach

This paper presents a new method to track both the face pose and the fac...
03/31/2020

StyleRig: Rigging StyleGAN for 3D Control over Portrait Images

StyleGAN generates photorealistic portrait images of faces with eyes, te...