Global-to-local Expression-aware Embeddings for Facial Action Unit Detection

by   Rudong An, et al.

Expressions and facial action units (AUs) are two levels of facial behavior descriptors. Expression auxiliary information has been widely used to improve the AU detection performance. However, most existing expression representations can only describe pre-determined discrete categories (e.g., Angry, Disgust, Happy, Sad, etc.) and cannot capture subtle expression transformations like AUs. In this paper, we propose a novel fine-grained Global Expression representation Encoder to capture subtle and continuous facial movements, to promote AU detection. To obtain such a global expression representation, we propose to train an expression embedding model on a large-scale expression dataset according to global expression similarity. Moreover, considering the local definition of AUs, it is essential to extract local AU features. Therefore, we design a Local AU Features Module to generate local facial features for each AU. Specifically, it consists of an AU feature map extractor and a corresponding AU mask extractor. First, the two extractors transform the global expression representation into AU feature maps and masks, respectively. Then, AU feature maps and their corresponding AU masks are multiplied to generate AU masked features focusing on local facial region. Finally, the AU masked features are fed into an AU classifier for judging the AU occurrence. Extensive experiment results demonstrate the superiority of our proposed method. Our method validly outperforms previous works and achieves state-of-the-art performances on widely-used face datasets, including BP4D, DISFA, and BP4D+.


page 1

page 4

page 9

page 12


LoRRaL: Facial Action Unit Detection Based on Local Region Relation Learning

End-to-end convolution representation learning has been proved to be ver...

Adaptive Local-Global Relational Network for Facial Action Units Recognition and Facial Paralysis Estimation

Facial action units (AUs) refer to a unique set of facial muscle movemen...

FG-Net: Facial Action Unit Detection with Generalizable Pyramidal Features

Automatic detection of facial Action Units (AUs) allows for objective fa...

Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution

The intensity estimation of facial action units (AUs) is challenging due...

Your "Attention" Deserves Attention: A Self-Diversified Multi-Channel Attention for Facial Action Analysis

Visual attention has been extensively studied for learning fine-grained ...

Local Relation Learning for Face Forgery Detection

With the rapid development of facial manipulation techniques, face forge...

AFNet-M: Adaptive Fusion Network with Masks for 2D+3D Facial Expression Recognition

2D+3D facial expression recognition (FER) can effectively cope with illu...

Please sign up or login with your details

Forgot password? Click here to reset