A Siamese Neural Network with Modified Distance Loss For Transfer Learning in Speech Emotion Recognition

06/04/2020
by   Kexin Feng, et al.
0

Automatic emotion recognition plays a significant role in the process of human computer interaction and the design of Internet of Things (IOT) technologies. Yet, a common problem in emotion recognition systems lies in the scarcity of reliable labels. By modeling pairwise differences between samples of interest, a Siamese network can help to mitigate this challenge since it requires fewer samples than traditional deep learning methods. In this paper, we propose a distance loss, which can be applied on the Siamese network fine-tuning, by optimizing the model based on the relevant distance between same and difference class pairs. Our system use samples from the source data to pre-train the weights of proposed Siamese neural network, which are fine-tuned based on the target data. We present an emotion recognition task that uses speech, since it is one of the most ubiquitous and frequently used bio-behavioral signals. Our target data comes from the RAVDESS dataset, while the CREMA-D and eNTERFACE'05 are used as source data, respectively. Our results indicate that the proposed distance loss is able to greatly benefit the fine-tuning process of Siamese network. Also, the selection of source data has more effect on the Siamese network performance compared to the number of frozen layers. These suggest the great potential of applying the Siamese network and modelling pairwise differences in the field of transfer learning for automatic emotion recognition.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/23/2020

Meta Transfer Learning for Emotion Recognition

Deep learning has been widely adopted in automatic emotion recognition a...
research
12/13/2015

Deep Learning-Based Image Kernel for Inductive Transfer

We propose a method to classify images from target classes with a small ...
research
09/07/2021

Few-shot Learning in Emotion Recognition of Spontaneous Speech Using a Siamese Neural Network with Adaptive Sample Pair Formation

Speech-based machine learning (ML) has been heralded as a promising solu...
research
02/16/2022

Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models

Automatic emotion recognition plays a key role in computer-human interac...
research
05/25/2018

Meta Transfer Learning for Facial Emotion Recognition

The use of deep learning techniques for automatic facial expression reco...
research
10/26/2022

Knowledge Transfer For On-Device Speech Emotion Recognition with Neural Structured Learning

Speech emotion recognition (SER) has been a popular research topic in hu...
research
10/17/2018

Learning an MR acquisition-invariant representation using Siamese neural networks

Generalization of voxelwise classifiers is hampered by differences betwe...

Please sign up or login with your details

Forgot password? Click here to reset