Transformer-based Multimodal Information Fusion for Facial Expression Analysis

03/23/2022
by   Wei Zhang, et al.
0

Facial expression analysis has been a crucial research problem in the computer vision area. With the recent development of deep learning techniques and large-scale in-the-wild annotated datasets, facial expression analysis is now aimed at challenges in real world settings. In this paper, we introduce our submission to CVPR2022 Competition on Affective Behavior Analysis in-the-wild (ABAW) that defines four competition tasks, including expression classification, action unit detection, valence-arousal estimation, and a multi-task-learning. The available multimodal information consist of spoken words, speech prosody, and visual expression in videos. Our work proposes four unified transformer-based network frameworks to create the fusion of the above multimodal information. The preliminary results on the official Aff-Wild2 dataset are reported and demonstrate the effectiveness of our proposed method.

READ FULL TEXT
research
07/08/2021

Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition

Automatic affective recognition has been an important research topic in ...
research
03/24/2022

An Ensemble Approach for Facial Expression Analysis in Video

Human emotions recognization contributes to the development of human-com...
research
03/24/2022

Expression Classification using Concatenation of Deep Neural Network for the 3rd ABAW3 Competition

For computers to recognize human emotions, expression classification is ...
research
09/20/2021

MFEViT: A Robust Lightweight Transformer-based Network for Multimodal 2D+3D Facial Expression Recognition

Vision transformer (ViT) has been widely applied in many areas due to it...
research
03/19/2023

TempT: Temporal consistency for Test-time adaptation

In this technical report, we introduce TempT, a novel method for test ti...
research
06/01/2023

A deep-learning approach to early identification of suggested sexual harassment from videos

Sexual harassment, sexual abuse, and sexual violence are prevalent probl...
research
03/11/2019

The Truth and Nothing but the Truth: Multimodal Analysis for Deception Detection

We propose a data-driven method for automatic deception detection in rea...

Please sign up or login with your details

Forgot password? Click here to reset