Attending to Emotional Narratives

07/08/2019
by   Zhengxuan Wu, et al.
0

Attention mechanisms in deep neural networks have achieved excellent performance on sequence-prediction tasks. Here, we show that these recently-proposed attention-based mechanisms---in particular, the Transformer with its parallelizable self-attention layers, and the Memory Fusion Network with attention across modalities and time---also generalize well to multimodal time-series emotion recognition. Using a recently-introduced dataset of emotional autobiographical narratives, we adapt and apply these two attention mechanisms to predict emotional valence over time. Our models perform extremely well, in some cases reaching a performance comparable with human raters. We end with a discussion of the implications of attention mechanisms to affective computing.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/22/2019

Modeling emotion in complex stories: the Stanford Emotional Narratives Dataset

Human emotions unfold over time, and more affective computing research h...
research
02/20/2023

Knowledge-aware Bayesian Co-attention for Multimodal Emotion Recognition

Multimodal emotion recognition is a challenging research area that aims ...
research
06/22/2021

Key-Sparse Transformer with Cascaded Cross-Attention Block for Multimodal Speech Emotion Recognition

Speech emotion recognition is a challenging and important research topic...
research
02/03/2021

Speech Emotion Recognition with Multiscale Area Attention and Data Augmentation

In Speech Emotion Recognition (SER), emotional characteristics often app...
research
03/24/2022

Continuous-Time Audiovisual Fusion with Recurrence vs. Attention for In-The-Wild Affect Recognition

In this paper, we present our submission to 3rd Affective Behavior Analy...
research
06/23/2023

Cross-Language Speech Emotion Recognition Using Multimodal Dual Attention Transformers

Despite the recent progress in speech emotion recognition (SER), state-o...
research
12/20/2019

SensAI+Expanse Adaptation on Human Behaviour Towards Emotional Valence Prediction

An agent, artificial or human, must be continuously adjusting its behavi...

Please sign up or login with your details

Forgot password? Click here to reset