DeepAI AI Chat
Log In Sign Up

Self-supervised Learning of Detailed 3D Face Reconstruction

10/25/2019
by   Yajing Chen, et al.
58

In this paper, we present an end-to-end learning framework for detailed 3D face reconstruction from a single image. Our approach uses a 3DMM-based coarse model and a displacement map in UV-space to represent a 3D face. Unlike previous work addressing the problem, our learning framework does not require supervision of surrogate ground-truth 3D models computed with traditional approaches. Instead, we utilize the input image itself as supervision during learning. In the first stage, we combine a photometric loss and a facial perceptual loss between the input face and the rendered face, to regress a 3DMM-based coarse model. In the second stage, both the input image and the regressed texture of the coarse model are unwrapped into UV-space, and then sent through an image-toimage translation network to predict a displacement map in UVspace. The displacement map and the coarse model are used to render a final detailed face, which again can be compared with the original input image to serve as a photometric loss for the second stage. The advantage of learning displacement map in UV-space is that face alignment can be explicitly done during the unwrapping, thus facial details are easier to learn from large amount of data. Extensive experiments demonstrate the superiority of the proposed method over previous work.

READ FULL TEXT

page 1

page 4

page 6

page 8

page 9

page 10

08/01/2017

Learning to Hallucinate Face Images via Component Generation and Enhancement

We propose a two-stage method for face hallucination. First, we generate...
11/16/2021

Self-supervised High-fidelity and Re-renderable 3D Facial Reconstruction from a Single Image

Reconstructing high-fidelity 3D facial texture from a single image is a ...
06/06/2021

Alpha Matte Generation from Single Input for Portrait Matting

Portrait matting is an important research problem with a wide range of a...
08/11/2021

SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recovery

We present SIDER(Single-Image neural optimization for facial geometric D...
04/25/2023

AVFace: Towards Detailed Audio-Visual 4D Face Reconstruction

In this work, we present a multimodal solution to the problem of 4D face...
12/15/2020

FCFR-Net: Feature Fusion based Coarse-to-Fine Residual Learning for Monocular Depth Completion

Depth completion aims to recover a dense depth map from a sparse depth m...
04/04/2019

Siamese Encoding and Alignment by Multiscale Learning with Self-Supervision

We propose a method of aligning a source image to a target image, where ...