StyleRig: Rigging StyleGAN for 3D Control over Portrait Images

03/31/2020
by   Ayush Tewari, et al.
0

StyleGAN generates photorealistic portrait images of faces with eyes, teeth, hair and context (neck, shoulders, background), but lacks a rig-like control over semantic face parameters that are interpretable in 3D, such as face pose, expressions, and scene illumination. Three-dimensional morphable face models (3DMMs) on the other hand offer control over the semantic parameters, but lack photorealism when rendered and only model the face interior, not other parts of a portrait image (hair, mouth interior, background). We present the first method to provide a face rig-like control over a pretrained and fixed StyleGAN via a 3DMM. A new rigging network, RigNet is trained between the 3DMM's semantic parameters and StyleGAN's input. The network is trained in a self-supervised manner, without the need for manual annotations. At test time, our method generates portrait images with the photorealism of StyleGAN and provides explicit control over the 3D semantic parameters of the face.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 8

page 11

page 12

research
08/31/2020

GIF: Generative Interpretable Faces

Photo-realistic visualization and animation of expressive human faces ha...
research
04/03/2019

ICface: Interpretable and Controllable Face Reenactment Using GANs

This paper presents a generic face animator that is able to control the ...
research
03/20/2019

Face Detection in Repeated Settings

Face detection is an important first step before face verification and r...
research
11/13/2018

Pose Invariant 3D Face Reconstruction

3D face reconstruction is an important task in the field of computer vis...
research
07/27/2018

X2Face: A network for controlling face generation by using images, audio, and pose codes

The objective of this paper is a neural network model that controls the ...
research
03/27/2023

FaceLit: Neural 3D Relightable Faces

We propose a generative framework, FaceLit, capable of generating a 3D f...
research
02/07/2019

FDDB-360: Face Detection in 360-degree Fisheye Images

360-degree cameras offer the possibility to cover a large area, for exam...

Please sign up or login with your details

Forgot password? Click here to reset