Faster Than Real-time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses

by   Chandrasekhar Bhagavatula, et al.

Facial alignment involves finding a set of landmark points on an image with a known semantic meaning. However, this semantic meaning of landmark points is often lost in 2D approaches where landmarks are either moved to visible boundaries or ignored as the pose of the face changes. In order to extract consistent alignment points across large poses, the 3D structure of the face must be considered in the alignment step. However, extracting a 3D structure from a single 2D image usually requires alignment in the first place. We present our novel approach to simultaneously extract the 3D shape of the face and the semantically consistent 2D alignment through a 3D Spatial Transformer Network (3DSTN) to model both the camera projection matrix and the warping parameters of a 3D model. By utilizing a generic 3D model and a Thin Plate Spline (TPS) warping function, we are able to generate subject specific 3D shapes without the need for a large 3D shape basis. In addition, our proposed network can be trained in an end-to-end framework on entirely synthetic data from the 300W-LP dataset. Unlike other 3D methods, our approach only requires one pass through the network resulting in a faster than real-time alignment. Evaluations of our model on the Annotated Facial Landmarks in the Wild (AFLW) and AFLW2000-3D datasets show our method achieves state-of-the-art performance over other 3D approaches to alignment.



page 1

page 2

page 4

page 6


Dense Face Alignment

Face alignment is a classic problem in the computer vision field. Previo...

Convolutional Point-set Representation: A Convolutional Bridge Between a Densely Annotated Image and 3D Face Alignment

We present a robust method for estimating the facial pose and shape info...

LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood

Modern face alignment methods have become quite accurate at predicting t...

Semantic Alignment: Finding Semantically Consistent Ground-truth for Facial Landmark Detection

Recently, deep learning based facial landmark detection has achieved gre...

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

We propose a straightforward method that simultaneously reconstructs the...

Face Alignment in Full Pose Range: A 3D Total Solution

Face alignment, which fits a face model to an image and extracts the sem...

Look at Boundary: A Boundary-Aware Face Alignment Algorithm

We present a novel boundary-aware face alignment algorithm by utilising ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.