Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz

12/07/2017
by   Ayush Tewari, et al.
0

The reconstruction of dense 3D models of face geometry and appearance from a single image is highly challenging and ill-posed. To constrain the problem, many approaches rely on strong priors, such as parametric face models learned from limited 3D scan data. However, prior models restrict generalization of the true diversity in facial geometry, skin reflectance and illumination. To alleviate this problem, we present the first approach that jointly learns 1) a regressor for face shape, expression, reflectance and illumination on the basis of 2) a concurrently learned parametric face model. Our multi-level face model combines the advantage of 3D Morphable Models for regularization with the out-of-space generalization of a learned corrective space. We train end-to-end on in-the-wild images without dense annotations by fusing a convolutional encoder with a differentiable expert-designed renderer and a self-supervised training loss, both defined at multiple detail levels. Our approach compares favorably to the state-of-the-art in terms of reconstruction quality, better generalizes to real world faces, and runs at over 250 Hz.

READ FULL TEXT

page 6

page 7

page 8

page 12

page 13

page 14

page 15

page 16

research
10/09/2020

Learning 3D Face Reconstruction with a Pose Guidance Network

We present a self-supervised learning approach to learning monocular 3D ...
research
10/04/2020

Learning Complete 3D Morphable Face Models from Images and Videos

Most 3D face reconstruction methods rely on 3D morphable models, which d...
research
12/18/2018

FML: Face Model Learning from Videos

Monocular image-based 3D reconstruction of faces is a long-standing prob...
research
03/30/2017

MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction

In this work we propose a novel model-based deep convolutional autoencod...
research
08/24/2020

Monocular Reconstruction of Neural Face Reflectance Fields

The reflectance field of a face describes the reflectance properties res...
research
08/28/2019

BioFaceNet: Deep Biophysical Face Image Interpretation

In this paper we present BioFaceNet, a deep CNN that learns to decompose...
research
10/07/2020

A Human Ear Reconstruction Autoencoder

The ear, as an important part of the human head, has received much less ...

Please sign up or login with your details

Forgot password? Click here to reset