MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction

03/30/2017
by   Ayush Tewari, et al.
0

In this work we propose a novel model-based deep convolutional autoencoder that addresses the highly challenging problem of reconstructing a 3D human face from a single in-the-wild color image. To this end, we combine a convolutional encoder network with an expert-designed generative model that serves as decoder. The core innovation is our new differentiable parametric decoder that encapsulates image formation analytically based on a generative model. Our decoder takes as input a code vector with exactly defined semantic meaning that encodes detailed face pose, shape, expression, skin reflectance and scene illumination. Due to this new way of combining CNN-based with model-based face reconstruction, the CNN-based encoder learns to extract semantically meaningful parameters from a single monocular input image. For the first time, a CNN encoder and an expert-designed generative model can be trained end-to-end in an unsupervised manner, which renders training on very large (unlabeled) real world data feasible. The obtained reconstructions compare favorably to current state-of-the-art approaches in terms of quality and richness of representation.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 7

page 8

research
12/07/2017

Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz

The reconstruction of dense 3D models of face geometry and appearance fr...
research
08/28/2019

BioFaceNet: Deep Biophysical Face Image Interpretation

In this paper we present BioFaceNet, a deep CNN that learns to decompose...
research
03/31/2017

InverseFaceNet: Deep Single-Shot Inverse Face Rendering From A Single Image

We introduce InverseFaceNet, a deep convolutional inverse rendering fram...
research
02/09/2019

3D Hand Shape and Pose from Images in the Wild

We present in this work the first end-to-end deep learning based method ...
research
03/29/2021

Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing

Robust face reconstruction from monocular image in general lighting cond...
research
10/25/2021

Learning Continuous Face Representation with Explicit Functions

How to represent a face pattern? While it is presented in a continuous w...
research
05/06/2021

Inverting Generative Adversarial Renderer for Face Reconstruction

Given a monocular face image as input, 3D face geometry reconstruction a...

Please sign up or login with your details

Forgot password? Click here to reset