Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

05/16/2021
by   Baris Gecer, et al.
8

A lot of work has been done towards reconstructing the 3D facial structure from single images by capitalizing on the power of Deep Convolutional Neural Networks (DCNNs). In the recent works, the texture features either correspond to components of a linear texture space or are learned by auto-encoders directly from in-the-wild images. In all cases, the quality of the facial texture reconstruction is still not capable of modeling facial texture with high-frequency details. In this paper, we take a radically different approach and harness the power of Generative Adversarial Networks (GANs) and DCNNs in order to reconstruct the facial texture and shape from single images. That is, we utilize GANs to train a very powerful facial texture prior from a large-scale 3D texture dataset. Then, we revisit the original 3D Morphable Models (3DMMs) fitting making use of non-linear optimization to find the optimal latent parameters that best reconstruct the test image but under a new perspective. In order to be robust towards initialisation and expedite the fitting process, we propose a novel self-supervised regression based approach. We demonstrate excellent results in photorealistic and identity preserving 3D face reconstructions and achieve for the first time, to the best of our knowledge, facial texture reconstruction with high-frequency details.

READ FULL TEXT

page 2

page 4

page 7

page 8

page 10

page 11

page 12

page 13

02/15/2019

GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction

In the past few years, a lot of work has been done towards reconstructin...
12/30/2020

OSTeC: One-Shot Texture Completion

The last few years have witnessed the great success of non-linear genera...
03/30/2020

AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"

Over the last years, with the advent of Generative Adversarial Networks ...
04/06/2019

Dense 3D Face Decoding over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoders

3D Morphable Models (3DMMs) are statistical models that represent facial...
05/01/2019

3DFaceGAN: Adversarial Nets for 3D Face Representation, Generation, and Translation

Over the past few years, Generative Adversarial Networks (GANs) have gar...
12/07/2019

Digital Twin: Acquiring High-Fidelity 3D Avatar from a Single Image

We present an approach to generate high fidelity 3D face avatar with a h...
12/21/2017

Smart, Sparse Contours to Represent and Edit Images

We study the problem of reconstructing an image from information stored ...