An Efficient Multitask Neural Network for Face Alignment, Head Pose Estimation and Face Tracking

by   Jiahao Xia, et al.

While convolutional neural networks (CNNs) have significantly boosted the performance of face related algorithms, maintaining accuracy and efficiency simultaneously in practical use remains challenging. Recent study shows that using a cascade of hourglass modules which consist of a number of bottom-up and top-down convolutional layers can extract facial structural information for face alignment to improve accuracy. However, previous studies have shown that features produced by shallow convolutional layers are highly correspond to edges. These features could be directly used to provide the structural information without addition cost. Motivated by this intuition, we propose an efficient multitask face alignment, face tracking and head pose estimation network (ATPN). Specifically, we introduce a shortcut connection between shallow-layer features and deep-layer features to provide the structural information for face alignment and apply the CoordConv to the last few layers to provide coordinate information. The predicted facial landmarks enable us to generate a cheap heatmap which contains both geometric and appearance information for head pose estimation and it also provides attention clues for face tracking. Moreover, the face tracking task saves us the face detection procedure for each frame, which is significant to boost performance for video-based tasks. The proposed framework is evaluated on four benchmark datasets, WFLW, 300VW, WIDER Face and 300W-LP. The experimental results show that the ATPN achieves improved performance compared to previous state-of-the-art methods while having less number of parameters and FLOPS.


page 2

page 4

page 10

page 11


3D Face Alignment Through Fusion of Head Pose Information and Features

The ability of humans to infer head poses from face shapes, and vice ver...

HoloFace: Augmenting Human-to-Human Interactions on HoloLens

We present HoloFace, an open-source framework for face alignment, head p...

Convolutional Point-set Representation: A Convolutional Bridge Between a Densely Annotated Image and 3D Face Alignment

We present a robust method for estimating the facial pose and shape info...

Simultaneous face detection and 360 degree headpose estimation

With many practical applications in human life, including manufacturing ...

Multi-modal Face Pose Estimation with Multi-task Manifold Deep Learning

Human face pose estimation aims at estimating the gazing direction or he...

Multistage Model for Robust Face Alignment Using Deep Neural Networks

An ability to generalize unconstrained conditions such as severe occlusi...

Pose-Invariant Face Alignment with a Single CNN

Face alignment has witnessed substantial progress in the last decade. On...

Please sign up or login with your details

Forgot password? Click here to reset