FML: Face Model Learning from Videos

12/18/2018
by   Ayush Tewari, et al.
30

Monocular image-based 3D reconstruction of faces is a long-standing problem in computer vision. Since image data is a 2D projection of a 3D face, the resulting depth ambiguity makes the problem ill-posed. Most existing methods rely on data-driven priors that are built from limited 3D face scans. In contrast, we propose multi-frame video-based self-supervised training of a deep network that (i) learns a face identity model both in shape and appearance while (ii) jointly learning to reconstruct 3D faces. Our face model is learned using only corpora of in-the-wild video clips collected from the Internet. This virtually endless source of training data enables learning of a highly general 3D face model. In order to achieve this, we propose a novel multi-frame consistency loss that ensures consistent shape and appearance across multiple frames of a subject's face, thus minimizing depth ambiguity. At test time we can use an arbitrary number of frames, so that we can perform both monocular as well as multi-frame reconstruction.

READ FULL TEXT

page 7

page 8

page 12

page 13

page 14

page 15

page 16

page 17

research
12/07/2017

Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz

The reconstruction of dense 3D models of face geometry and appearance fr...
research
07/24/2020

Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency

Recent learning-based approaches, in which models are trained by single-...
research
04/08/2021

Riggable 3D Face Reconstruction via In-Network Optimization

This paper presents a method for riggable 3D face reconstruction from mo...
research
10/04/2020

Learning Complete 3D Morphable Face Models from Images and Videos

Most 3D face reconstruction methods rely on 3D morphable models, which d...
research
09/29/2021

Identity-Expression Ambiguity in 3D Morphable Face Models

3D Morphable Models are a class of generative models commonly used to mo...
research
04/27/2017

Deep Face Deblurring

Blind deblurring consists a long studied task, however the outcomes of g...
research
08/22/2017

What does 2D geometric information really tell us about 3D face shape?

A face image contains geometric cues in the form of configurational info...

Please sign up or login with your details

Forgot password? Click here to reset