Two-level Attention with Two-stage Multi-task Learning for Facial Emotion Recognition

11/29/2018
by   Xiaohua Wang, et al.
0

Compared with facial emotion recognition on categorical model, the dimensional emotion recognition can describe numerous emotions of the real world more accurately. Most prior works of dimensional emotion estimation only considered laboratory data and used video, speech or other multi-modal features. The effect of these methods applied on static images in the real world is unknown. In this paper, a two-level attention with two-stage multi-task learning (2Att-2Mt) framework is proposed for facial emotion estimation on only static images. Firstly, the features of corresponding region(position-level features) are extracted and enhanced automatically by first-level attention mechanism. In the following, we utilize Bi-directional Recurrent Neural Network(Bi-RNN) with self-attention(second-level attention) to make full use of the relationship features of different layers(layer-level features) adaptively. Owing to the inherent complexity of dimensional emotion recognition, we propose a two-stage multi-task learning structure to exploited categorical representations to ameliorate the dimensional representations and estimate valence and arousal simultaneously in view of the correlation of the two targets. The quantitative results conducted on AffectNet dataset show significant advancement on Concordance Correlation Coefficient(CCC) and Root Mean Square Error(RMSE), illustrating the superiority of the proposed framework. Besides, extensive comparative experiments have also fully demonstrated the effectiveness of different components.

READ FULL TEXT
research
06/08/2021

Efficient Speech Emotion Recognition Using Multi-Scale CNN and Attention

Emotion recognition from speech is a challenging task. Re-cent advances ...
research
11/02/2020

Multimodal Continuous Emotion Recognition using Deep Multi-Task Learning with Correlation Loss

In this study, we focus on continuous emotion recognition using body mot...
research
03/29/2019

Attention-Augmented End-to-End Multi-Task Learning for Emotion Prediction from Speech

Despite the increasing research interest in end-to-end learning systems ...
research
10/03/2019

Exploiting multi-CNN features in CNN-RNN based Dimensional Emotion Recognition on the OMG in-the-wild Dataset

This paper presents a novel CNN-RNN based approach, which exploits multi...
research
10/19/2020

Facial Emotion Recognition with Noisy Multi-task Annotations

Human emotions can be inferred from facial expressions. However, the ann...
research
04/27/2019

Human-Centered Emotion Recognition in Animated GIFs

As an intuitive way of expression emotion, the animated Graphical Interc...
research
03/03/2018

An Ensemble Framework of Voice-Based Emotion Recognition System for Films and TV Programs

Employing voice-based emotion recognition function in artificial intelli...

Please sign up or login with your details

Forgot password? Click here to reset