End-to-end Multimodal Emotion and Gender Recognition with Dynamic Weights of Joint Loss

09/04/2018
by   Myungsu Chae, et al.
0

Multi-task learning (MTL) is one of the method for improving generalizability of multiple tasks. In order to perform multiple classification tasks with one neural network model, the losses of each task should be combined. Previous studies have mostly focused on prediction of multiple tasks using joint loss with static weights for training model. Choosing weights between tasks have not taken any considerations while it is set by uniformly or empirically. In this study, we propose a method to make joint loss using dynamic weights to improve total performance not an individual performance of tasks, and apply this method to end-to-end multimodal emotion and gender recognition model using audio and video data. This approach provides proper weights for each loss of the tasks when training ends. In our experiment, a performance of emotion and gender recognition with proposed method shows lower joint loss which is computed as negative log-likelihood than the one with static weights of joint loss. Also, our proposed model shows better generalizability than compared models. In our best knowledge, this research shows the strength of dynamic weights of joint loss for maximizing total performance at first in emotion and gender recognition task.

READ FULL TEXT
research
09/04/2018

End-to-end Multimodal Emotion and Gender Recognition with Dynamic Joint Loss Weights

Multi-task learning is a method for improving the generalizability of mu...
research
09/03/2020

Multi-Loss Weighting with Coefficient of Variations

Many interesting tasks in machine learning and computer vision are learn...
research
06/18/2022

Redundancy Reduction Twins Network: A Training framework for Multi-output Emotion Regression

In this paper, we propose the Redundancy Reduction Twins Network (RRTN),...
research
08/13/2017

Towards Speech Emotion Recognition "in the wild" using Aggregated Corpora and Deep Multi-Task Learning

One of the challenges in Speech Emotion Recognition (SER) "in the wild" ...
research
11/05/2016

A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks

Transfer and multi-task learning have traditionally focused on either a ...
research
03/29/2019

Attention-Augmented End-to-End Multi-Task Learning for Emotion Prediction from Speech

Despite the increasing research interest in end-to-end learning systems ...
research
05/27/2023

Understanding Emotion Valence is a Joint Deep Learning Task

The valence analysis of speakers' utterances or written posts helps to u...

Please sign up or login with your details

Forgot password? Click here to reset