REALY: Rethinking the Evaluation of 3D Face Reconstruction

by   Zenghao Chai, et al.

The evaluation of 3D face reconstruction results typically relies on a rigid shape alignment between the estimated 3D model and the ground-truth scan. We observe that aligning two shapes with different reference points can largely affect the evaluation results. This poses difficulties for precisely diagnosing and improving a 3D face reconstruction method. In this paper, we propose a novel evaluation approach with a new benchmark REALY, consists of 100 globally aligned face scans with accurate facial keypoints, high-quality region masks, and topology-consistent meshes. Our approach performs region-wise shape alignment and leads to more accurate, bidirectional correspondences during computing the shape errors. The fine-grained, region-wise evaluation results provide us detailed understandings about the performance of state-of-the-art 3D face reconstruction methods. For example, our experiments on single-image based reconstruction methods reveal that DECA performs the best on nose regions, while GANFit performs better on cheek regions. Besides, a new and high-quality 3DMM basis, HIFI3D++, is further derived using the same procedure as we construct REALY to align and retopologize several 3D face datasets. We will release REALY, HIFI3D++, and our new evaluation pipeline at


page 19

page 22

page 23

page 26

page 27

page 29


Evaluation of Dense 3D Reconstruction from 2D Face Images in the Wild

This paper investigates the evaluation of dense 3D face reconstruction f...

JIFF: Jointly-aligned Implicit Face Function for High Quality Single View Clothed Human Reconstruction

This paper addresses the problem of single view 3D human reconstruction....

Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set

Recently, deep learning based 3D face reconstruction methods have shown ...

3D Face Hallucination from a Single Depth Frame

We present an algorithm that takes a single frame of a person's face fro...

Adaptive 3D Face Reconstruction from a Single Image

3D face reconstruction from a single image is a challenging problem, esp...

FSNet: An Identity-Aware Generative Model for Image-based Face Swapping

This paper presents FSNet, a deep generative model for image-based face ...

Pruning-based Topology Refinement of 3D Mesh using a 2D Alpha Mask

Image-based 3D reconstruction has increasingly stunning results over the...

Please sign up or login with your details

Forgot password? Click here to reset