GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction

by   Baris Gecer, et al.
Imperial College London

In the past few years, a lot of work has been done towards reconstructing the 3D facial structure from single images by capitalizing on the power of Deep Convolutional Neural Networks (DCNNs). In the most recent works, differentiable renderers were employed in order to learn the relationship between the facial identity features and the parameters of a 3D morphable model for shape and texture. The texture features either correspond to components of a linear texture space or are learned by auto-encoders directly from in-the-wild images. In all cases, the quality of the facial texture reconstruction of the state-of-the-art methods is still not capable of modelling textures in high fidelity. In this paper, we take a radically different approach and harness the power of Generative Adversarial Networks (GANs) and DCNNs in order to reconstruct the facial texture and shape from single images. That is, we utilize GANs to train a very powerful generator of facial texture in UV space. Then, we revisit the original 3D Morphable Models (3DMMs) fitting approaches making use of non-linear optimization to find the optimal latent parameters that best reconstruct the test image but under a new perspective. We optimize the parameters with the supervision of pretrained deep identity features through our end-to-end differentiable framework. We demonstrate excellent results in photorealistic and identity preserving 3D face reconstructions and achieve for the first time, to the best of our knowledge, facial texture reconstruction with high-frequency details.


page 1

page 3

page 6

page 7

page 8

page 11

page 12

page 13


Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

A lot of work has been done towards reconstructing the 3D facial structu...

3D Face Morphable Models "In-the-Wild"

3D Morphable Models (3DMMs) are powerful statistical models of 3D facial...

Digital Twin: Acquiring High-Fidelity 3D Avatar from a Single Image

We present an approach to generate high fidelity 3D face avatar with a h...

Nonlinear 3D Face Morphable Model

As a classic statistical model of 3D facial shape and texture, 3D Morpha...

Dense 3D Face Decoding over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoders

3D Morphable Models (3DMMs) are statistical models that represent facial...

StyleUV: Diverse and High-fidelity UV Map Generative Model

Reconstructing 3D human faces in the wild with the 3D Morphable Model (3...

Learning Oracle Attention for High-fidelity Face Completion

High-fidelity face completion is a challenging task due to the rich and ...

Please sign up or login with your details

Forgot password? Click here to reset