A Self-Supervised Bootstrap Method for Single-Image 3D Face Reconstruction

12/14/2018
by   Yifan Xing, et al.
16

State-of-the-art methods for 3D reconstruction of faces from a single image require 2D-3D pairs of ground-truth data for supervision. Such data is costly to acquire, and most datasets available in the literature are restricted to pairs for which the input 2D images depict faces in a near fronto-parallel pose. Therefore, many data-driven methods for single-image 3D facial reconstruction perform poorly on profile and near-profile faces. We propose a method to improve the performance of single-image 3D facial reconstruction networks by utilizing the network to synthesize its own training data for fine-tuning, comprising: (i) single-image 3D reconstruction of faces in near-frontal images without ground-truth 3D shape; (ii) application of a rigid-body transformation to the reconstructed face model; (iii) rendering of the face model from new viewpoints; and (iv) use of the rendered image and corresponding 3D reconstruction as additional data for supervised fine-tuning. The new 2D-3D pairs thus produced have the same high-quality observed for near fronto-parallel reconstructions, thereby nudging the network towards more uniform performance as a function of the viewing angle of input faces. Application of the proposed technique to the fine-tuning of a state-of-the-art single-image 3D-reconstruction network for faces demonstrates the usefulness of the method, with particularly significant gains for profile or near-profile views.

READ FULL TEXT

page 2

page 5

page 6

page 8

research
10/13/2021

A Literature Review of 3D Face Reconstruction From a Single Image

This paper is a brief survey of the recent literature on 3D face reconst...
research
10/19/2020

A Backbone Replaceable Fine-tuning Network for Stable Face Alignment

Heatmap regression based face alignment algorithms have achieved promine...
research
12/23/2021

Pose Adaptive Dual Mixup for Few-Shot Single-View 3D Reconstruction

We present a pose adaptive few-shot learning procedure and a two-stage d...
research
03/31/2017

InverseFaceNet: Deep Single-Shot Inverse Face Rendering From A Single Image

We introduce InverseFaceNet, a deep convolutional inverse rendering fram...
research
04/04/2019

3D Face Reconstruction Using Color Photometric Stereo with Uncalibrated Near Point Lights

We present a new color photometric stereo (CPS) method that can recover ...
research
01/05/2016

Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis

An important problem for both graphics and vision is to synthesize novel...
research
07/15/2021

Single-image Full-body Human Relighting

We present a single-image data-driven method to automatically relight im...

Please sign up or login with your details

Forgot password? Click here to reset