M^3T: Multi-Modal Continuous Valence-Arousal Estimation in the Wild

02/07/2020
by   Yuan-Hang Zhang, et al.
0

This report describes a multi-modal multi-task (M^3T) approach underlying our submission to the valence-arousal estimation track of the Affective Behavior Analysis in-the-wild (ABAW) Challenge, held in conjunction with the IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2020. In the proposed M^3T framework, we fuse both visual features from videos and acoustic features from the audio tracks to estimate the valence and arousal. The spatio-temporal visual features are extracted with a 3D convolutional network and a bidirectional recurrent neural network. Considering the correlations between valence / arousal, emotions, and facial actions, we also explores mechanisms to benefit from other tasks. We evaluated the M^3T framework on the validation set provided by ABAW and it significantly outperforms the baseline method.

READ FULL TEXT
research
02/13/2020

Emotion Recognition for In-the-wild Videos

This paper is a brief introduction to our submission to the seven basic ...
research
03/24/2022

Multi-modal Emotion Estimation for in-the-wild Videos

In this paper, we briefly introduce our submission to the Valence-Arousa...
research
07/02/2018

Multi-modal Egocentric Activity Recognition using Audio-Visual Features

Egocentric activity recognition in first-person videos has an increasing...
research
02/26/2020

Multi-Modal Continuous Valence And Arousal Prediction in the Wild Using Deep 3D Features and Sequence Modeling

Continuous affect prediction in the wild is a very interesting problem a...
research
09/19/2023

FoleyGen: Visually-Guided Audio Generation

Recent advancements in audio generation have been spurred by the evoluti...
research
12/12/2018

A Multimodal LSTM for Predicting Listener Empathic Responses Over Time

People naturally understand the emotions of-and often also empathize wit...
research
04/30/2018

OMG - Emotion Challenge Solution

This short paper describes our solution to the 2018 IEEE World Congress ...

Please sign up or login with your details

Forgot password? Click here to reset