LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood

by   Abhinav Kumar, et al.

Modern face alignment methods have become quite accurate at predicting the locations of facial landmarks, but they do not typically estimate the uncertainty of their predicted locations nor predict whether landmarks are visible. In this paper, we present a novel framework for jointly predicting landmark locations, associated uncertainties of these predicted locations, and landmark visibilities. We model these as mixed random variables and estimate them using a deep network trained with our proposed Location, Uncertainty, and Visibility Likelihood (LUVLi) loss. In addition, we release an entirely new labeling of a large face alignment dataset with over 19,000 face images in a full range of head poses. Each face is manually labeled with the ground-truth locations of 68 landmarks, with the additional information of whether each landmark is unoccluded, self-occluded (due to extreme head poses), or externally occluded. Not only does our joint estimation yield accurate estimates of the uncertainty of predicted landmark locations, but it also yields state-of-the-art estimates for the landmark locations themselves on multiple standard face alignment datasets. Our method's estimates of the uncertainty of predicted landmark locations could be used to automatically identify input images on which face alignment fails, which can be critical for downstream tasks.


page 1

page 17

page 18


Pose-Invariant 3D Face Alignment

Face alignment aims to estimate the locations of a set of landmarks for ...

Generating Talking Face Landmarks from Speech

The presence of a corresponding talking face has been shown to significa...

Deep Alignment Network: A convolutional neural network for robust face alignment

In this paper, we propose Deep Alignment Network (DAN), a robust face al...

Faster Than Real-time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses

Facial alignment involves finding a set of landmark points on an image w...

Mutual Information Maximization on Disentangled Representations for Differential Morph Detection

In this paper, we present a novel differential morph detection framework...

Inner Eye Canthus Localization for Human Body Temperature Screening

In this paper, we propose an automatic approach for localizing the inner...

Two-stage Convolutional Part Heatmap Regression for the 1st 3D Face Alignment in the Wild (3DFAW) Challenge

This paper describes our submission to the 1st 3D Face Alignment in the ...

Please sign up or login with your details

Forgot password? Click here to reset