Joint Action Unit localisation and intensity estimation through heatmap regression

05/09/2018
by   Enrique Sanchez, et al.
0

This paper proposes a supervised learning approach to jointly perform facial Action Unit (AU) localisation and intensity estimation. Contrary to previous works that try to learn an unsupervised representation of the Action Unit regions, we propose to directly and jointly estimate all AU intensities through heatmap regression, along with the location in the face where they cause visible changes. Our approach aims to learn a pixel-wise regression function returning a score per AU, which indicates an AU intensity at a given spatial location. Heatmap regression then generates an image, or channel, per AU, in which each pixel indicates the corresponding AU intensity. To generate the ground-truth heatmaps for a target AU, the facial landmarks are first estimated, and a 2D Gaussian is drawn around the points where the AU is known to cause changes. The amplitude and size of the Gaussian is determined by the intensity of the AU. We show that using a single Hourglass network suffices to attain new state of the art results, demonstrating the effectiveness of such a simple approach. The use of heatmap regression allows learning of a shared representation between AUs without the need to rely on latent representations, as these are implicitly learned from the data. We validate the proposed approach on the BP4D dataset, showing a modest improvement on recent, complex, techniques, as well as robustness against misalignment errors. We will release the code and the models to validate the experimental results.

READ FULL TEXT

page 2

page 9

research
04/14/2020

A Transfer Learning approach to Heatmap Regression for Action Unit intensity estimation

Action Units (AUs) are geometrically-based atomic facial muscle movement...
research
11/03/2020

Semi-supervised Facial Action Unit Intensity Estimation with Contrastive Learning

This paper tackles the challenging problem of estimating the intensity o...
research
10/28/2022

Facial Action Unit Detection and Intensity Estimation from Self-supervised Representation

As a fine-grained and local expression behavior measurement, facial acti...
research
04/20/2020

Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution

The intensity estimation of facial action units (AUs) is challenging due...
research
04/04/2019

Inferring Dynamic Representations of Facial Actions from a Still Image

Facial actions are spatio-temporal signals by nature, and therefore thei...
research
04/13/2020

Unsupervised Facial Action Unit Intensity Estimation via Differentiable Optimization

The automatic intensity estimation of facial action units (AUs) from a s...
research
05/24/2014

Improvements and Experiments of a Compact Statistical Background Model

Change detection plays an important role in most video-based application...

Please sign up or login with your details

Forgot password? Click here to reset