Disentanglement for audio-visual emotion recognition using multitask setup

02/11/2021
by   Raghuveer Peri, et al.
0

Deep learning models trained on audio-visual data have been successfully used to achieve state-of-the-art performance for emotion recognition. In particular, models trained with multitask learning have shown additional performance improvements. However, such multitask models entangle information between the tasks, encoding the mutual dependencies present in label distributions in the real world data used for training. This work explores the disentanglement of multimodal signal representations for the primary task of emotion recognition and a secondary person identification task. In particular, we developed a multitask framework to extract low-dimensional embeddings that aim to capture emotion specific information, while containing minimal information related to person identity. We evaluate three different techniques for disentanglement and report results of up to 13 recognition performance.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

09/06/2019

Learning Alignment for Multimodal Emotion Recognition from Speech

Speech emotion recognition is a challenging problem because human convey...
07/08/2020

Temporal aggregation of audio-visual modalities for emotion recognition

Emotion recognition has a pivotal role in affective computing and in hum...
04/18/2021

Why Should I Trust a Model is Private? Using Shifts in Model Explanation for Evaluating Privacy-Preserving Emotion Recognition Model

Privacy preservation is a crucial component of any real-world applicatio...
04/12/2019

Multimodal Speech Emotion Recognition and Ambiguity Resolution

Identifying emotion from speech is a non-trivial task pertaining to the ...
09/09/2021

Accounting for Variations in Speech Emotion Recognition with Nonparametric Hierarchical Neural Network

In recent years, deep-learning-based speech emotion recognition models h...
06/09/2020

Learning Functions to Study the Benefit of Multitask Learning

We study and quantify the generalization patterns of multitask learning ...
12/19/2019

Neural Simile Recognition with Cyclic Multitask Learning and Local Attention

Simile recognition is to detect simile sentences and to extract simile c...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.