A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognition

07/09/2021
by   Yue Jin, et al.
0

Analyzing human affect is vital for human-computer interaction systems. Most methods are developed in restricted scenarios which are not practical for in-the-wild settings. The Affective Behavior Analysis in-the-wild (ABAW) 2021 Contest provides a benchmark for this in-the-wild problem. In this paper, we introduce a multi-modal and multi-task learning method by using both visual and audio information. We use both AU and expression annotations to train the model and apply a sequence model to further extract associations between video frames. We achieve an AU score of 0.712 and an expression score of 0.477 on the validation set. These results demonstrate the effectiveness of our approach in improving model performance.

READ FULL TEXT

page 2

page 3

research
09/29/2020

A Multi-term and Multi-task Analyzing Framework for Affective Analysis in-the-wild

Human affective recognition is an important factor in human-computer int...
research
03/28/2022

Multi-model Ensemble Learning Method for Human Expression Recognition

Analysis of human affect plays a vital role in human-computer interactio...
research
07/20/2022

Multi-Task Learning for Emotion Descriptors Estimation at the fourth ABAW Challenge

Facial valence/arousal, expression and action unit are related tasks in ...
research
07/07/2021

Multi-modal Affect Analysis using standardized data within subjects in the Wild

Human affective recognition is an important factor in human-computer int...
research
02/21/2020

Affective Expression Analysis in-the-wild using Multi-Task Temporal Statistical Deep Learning Model

Affective behavior analysis plays an important role in human-computer in...
research
03/17/2023

Multi-modal Expression Recognition with Ensemble Method

This paper presents our submission to the Expression Classification Chal...
research
07/08/2021

Feature Pyramid Network for Multi-task Affective Analysis

Affective Analysis is not a single task, and the valence-arousal value, ...

Please sign up or login with your details

Forgot password? Click here to reset