DSFNet: Dual Space Fusion Network for Occlusion-Robust 3D Dense Face Alignment

05/19/2023
by   Heyuan Li, et al.
0

Sensitivity to severe occlusion and large view angles limits the usage scenarios of the existing monocular 3D dense face alignment methods. The state-of-the-art 3DMM-based method, directly regresses the model's coefficients, underutilizing the low-level 2D spatial and semantic information, which can actually offer cues for face shape and orientation. In this work, we demonstrate how modeling 3D facial geometry in image and model space jointly can solve the occlusion and view angle problems. Instead of predicting the whole face directly, we regress image space features in the visible facial region by dense prediction first. Subsequently, we predict our model's coefficients based on the regressed feature of the visible regions, leveraging the prior knowledge of whole face geometry from the morphable models to complete the invisible regions. We further propose a fusion network that combines the advantages of both the image and model space predictions to achieve high robustness and accuracy in unconstrained scenarios. Thanks to the proposed fusion module, our method is robust not only to occlusion and large pitch and roll view angles, which is the benefit of our image space approach, but also to noise and large yaw angles, which is the benefit of our model space method. Comprehensive evaluations demonstrate the superior performance of our method compared with the state-of-the-art methods. On the 3D dense face alignment task, we achieve 3.80 outperforms the state-of-the-art method by 5.5 https://github.com/lhyfst/DSFNet.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 8

research
06/06/2021

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Three-dimensional face dense alignment and reconstruction in the wild is...
research
03/21/2018

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

We propose a straightforward method that simultaneously reconstructs the...
research
04/11/2019

3D Dense Face Alignment via Graph Convolution Networks

Recently, 3D face reconstruction and face alignment tasks are gradually ...
research
04/02/2018

Face Alignment in Full Pose Range: A 3D Total Solution

Face alignment, which fits a face model to an image and extracts the sem...
research
10/19/2021

Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry

This work studies learning from a synergy process of 3D Morphable Models...
research
09/02/2020

3D Facial Geometry Recovery from a Depth View with Attention Guided Generative Adversarial Network

We present to recover the complete 3D facial geometry from a single dept...
research
08/18/2014

Learning Deep Representation for Face Alignment with Auxiliary Attributes

In this study, we show that landmark detection or face alignment task is...

Please sign up or login with your details

Forgot password? Click here to reset