Extending RNN-T-based speech recognition systems with emotion and language classification

07/28/2022
by   Zvi Kons, et al.
0

Speech transcription, emotion recognition, and language identification are usually considered to be three different tasks. Each one requires a different model with a different architecture and training process. We propose using a recurrent neural network transducer (RNN-T)-based speech-to-text (STT) system as a common component that can be used for emotion recognition and language identification as well as for speech recognition. Our work extends the STT system for emotion classification through minimal changes, and shows successful results on the IEMOCAP and MELD datasets. In addition, we demonstrate that by adding a lightweight component to the RNN-T module, it can also be used for language identification. In our evaluations, this new classifier demonstrates state-of-the-art accuracy for the NIST-LRE-07 dataset.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/23/2018

Evaluating Gammatone Frequency Cepstral Coefficients with Neural Networks for Emotion Recognition from Speech

Current approaches to speech emotion recognition focus on speech feature...
research
10/08/2021

Affective Burst Detection from Speech using Kernel-fusion Dilated Convolutional Neural Networks

As speech-interfaces are getting richer and widespread, speech emotion r...
research
05/30/2018

Context-aware Cascade Attention-based RNN for Video Emotion Recognition

Emotion recognition can provide crucial information about the user in ma...
research
01/27/2017

Emotion Recognition From Speech With Recurrent Neural Networks

In this paper the task of emotion recognition from speech is considered....
research
06/19/2018

The PRIORI Emotion Dataset: Linking Mood to Emotion Detected In-the-Wild

Bipolar Disorder is a chronic psychiatric illness characterized by patho...
research
11/14/2018

A Study of Language and Classifier-independent Feature Analysis for Vocal Emotion Recognition

Every speech signal carries implicit information about the emotions, whi...
research
06/10/2022

Nominal Metaphor Generation with Multitask Learning

Nominal metaphors are frequently used in human language and have been sh...

Please sign up or login with your details

Forgot password? Click here to reset