Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition?

10/10/2015
by   Pooya Khorrami, et al.
0

Despite being the appearance-based classifier of choice in recent years, relatively few works have examined how much convolutional neural networks (CNNs) can improve performance on accepted expression recognition benchmarks and, more importantly, examine what it is they actually learn. In this work, not only do we show that CNNs can achieve strong performance, but we also introduce an approach to decipher which portions of the face influence the CNN's predictions. First, we train a zero-bias CNN on facial expression data and achieve, to our knowledge, state-of-the-art performance on two expression recognition benchmarks: the extended Cohn-Kanade (CK+) dataset and the Toronto Face Dataset (TFD). We then qualitatively analyze the network by visualizing the spatial patterns that maximally excite different neurons in the convolutional layers and show how they resemble Facial Action Units (FAUs). Finally, we use the FAU labels provided in the CK+ dataset to verify that the FAUs observed in our filter visualizations indeed align with the subject's facial movements.

READ FULL TEXT

page 1

page 6

page 7

page 8

research
12/09/2016

Facial Expression Recognition using Convolutional Neural Networks: State of the Art

The ability to recognize facial expressions automatically enables novel ...
research
10/09/2017

Island Loss for Learning Discriminative Features in Facial Expression Recognition

Over the past few years, Convolutional Neural Networks (CNNs) have shown...
research
12/07/2018

Backdooring Convolutional Neural Networks via Targeted Weight Perturbations

We present a new type of backdoor attack that exploits a vulnerability o...
research
07/26/2017

Optimizing Filter Size in Convolutional Neural Networks for Facial Action Unit Recognition

Recognizing facial action units (AUs) during spontaneous facial displays...
research
07/17/2017

Incremental Boosting Convolutional Neural Network for Facial Action Unit Recognition

Recognizing facial action units (AUs) from spontaneous facial expression...
research
03/15/2018

Deep Structure Inference Network for Facial Action Unit Recognition

Facial expressions are combinations of basic components called Action Un...
research
10/17/2018

Coherence Constraints in Facial Expression Recognition

Recognizing facial expressions from static images or video sequences is ...

Please sign up or login with your details

Forgot password? Click here to reset