Attention-Augmented End-to-End Multi-Task Learning for Emotion Prediction from Speech

by   Zixing Zhang, et al.
Imperial College London

Despite the increasing research interest in end-to-end learning systems for speech emotion recognition, conventional systems either suffer from the overfitting due in part to the limited training data, or do not explicitly consider the different contributions of automatically learnt representations for a specific task. In this contribution, we propose a novel end-to-end framework which is enhanced by learning other auxiliary tasks and an attention mechanism. That is, we jointly train an end-to-end network with several different but related emotion prediction tasks, i.e., arousal, valence, and dominance predictions, to extract more robust representations shared among various tasks than traditional systems with the hope that it is able to relieve the overfitting problem. Meanwhile, an attention layer is implemented on top of the layers for each task, with the aim to capture the contribution distribution of different segment parts for each individual task. To evaluate the effectiveness of the proposed system, we conducted a set of experiments on the widely used database IEMOCAP. The empirical results show that the proposed systems significantly outperform corresponding baseline systems.


page 1

page 2

page 3

page 4


Two-level Attention with Two-stage Multi-task Learning for Facial Emotion Recognition

Compared with facial emotion recognition on categorical model, the dimen...

Unifying the Discrete and Continuous Emotion labels for Speech Emotion Recognition

Traditionally, in paralinguistic analysis for emotion detection from spe...

Distributionally Robust End-to-End Portfolio Construction

We propose an end-to-end distributionally robust system for portfolio co...

Multitask Learning from Augmented Auxiliary Data for Improving Speech Emotion Recognition

Despite the recent progress in speech emotion recognition (SER), state-o...

Improving End-to-End Text Image Translation From the Auxiliary Text Translation Task

End-to-end text image translation (TIT), which aims at translating the s...

Two-Aspect Information Fusion Model For ABAW4 Multi-task Challenge

In this paper, we propose the solution to the Multi-Task Learning (MTL) ...

End-to-end Multimodal Emotion and Gender Recognition with Dynamic Weights of Joint Loss

Multi-task learning (MTL) is one of the method for improving generalizab...

Please sign up or login with your details

Forgot password? Click here to reset