DeepAI
Log In Sign Up

End-to-end Multimodal Emotion and Gender Recognition with Dynamic Weights of Joint Loss

09/04/2018
by   Myungsu Chae, et al.
0

Multi-task learning (MTL) is one of the method for improving generalizability of multiple tasks. In order to perform multiple classification tasks with one neural network model, the losses of each task should be combined. Previous studies have mostly focused on prediction of multiple tasks using joint loss with static weights for training model. Choosing weights between tasks have not taken any considerations while it is set by uniformly or empirically. In this study, we propose a method to make joint loss using dynamic weights to improve total performance not an individual performance of tasks, and apply this method to end-to-end multimodal emotion and gender recognition model using audio and video data. This approach provides proper weights for each loss of the tasks when training ends. In our experiment, a performance of emotion and gender recognition with proposed method shows lower joint loss which is computed as negative log-likelihood than the one with static weights of joint loss. Also, our proposed model shows better generalizability than compared models. In our best knowledge, this research shows the strength of dynamic weights of joint loss for maximizing total performance at first in emotion and gender recognition task.

READ FULL TEXT
09/04/2018

End-to-end Multimodal Emotion and Gender Recognition with Dynamic Joint Loss Weights

Multi-task learning is a method for improving the generalizability of mu...
08/13/2017

Towards Speech Emotion Recognition "in the wild" using Aggregated Corpora and Deep Multi-Task Learning

One of the challenges in Speech Emotion Recognition (SER) "in the wild" ...
06/18/2022

Redundancy Reduction Twins Network: A Training framework for Multi-output Emotion Regression

In this paper, we propose the Redundancy Reduction Twins Network (RRTN),...
11/05/2016

A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks

Transfer and multi-task learning have traditionally focused on either a ...
07/13/2019

Multi-Task Semi-Supervised Adversarial Autoencoding for Speech Emotion

Despite the widespread use of supervised deep learning methods for affec...
03/29/2019

Attention-Augmented End-to-End Multi-Task Learning for Emotion Prediction from Speech

Despite the increasing research interest in end-to-end learning systems ...

Code Repositories

IROS2018_ws

End-to-end multimodal emotion and gender recognition with dynamic weights of joint loss


view repo