Linking emotions to behaviors through deep transfer learning

10/08/2019
by   Haoqi Li, et al.
0

Human behavior refers to the way humans act and interact. Understanding human behavior is a cornerstone of observational practice, especially in psychotherapy. An important cue of behavior analysis is the dynamical changes of emotions during the conversation. Domain experts integrate emotional information in a highly nonlinear manner, thus, it is challenging to explicitly quantify the relationship between emotions and behaviors. In this work, we employ deep transfer learning to analyze their inferential capacity and contextual importance. We first train a network to quantify emotions from acoustic signals and then use information from the emotion recognition network as features for behavior recognition. We treat this emotion-related information as behavioral primitives and further train higher level layers towards behavior quantification. Through our analysis, we find that emotion-related information is an important cue for behavior recognition. Further, we investigate the importance of emotional-context in the expression of behavior by constraining (or not) the neural networks' contextual view of the data. This demonstrates that the sequence of emotions is critical in behavior expression. To achieve these frameworks we employ hybrid architectures of convolutional networks and recurrent networks to extract emotion-related behavior primitives and facilitate automatic behavior recognition from speech.

READ FULL TEXT

page 1

page 6

page 7

research
01/24/2022

Do Smart Glasses Dream of Sentimental Visions? Deep Emotionship Analysis for Eyewear Devices

Emotion recognition in smart eyewear devices is highly valuable but chal...
research
10/23/2018

Classifying and Visualizing Emotions with Emotional DAN

Classification of human emotions remains an important and challenging ta...
research
01/31/2020

Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor

Automatic facial expression recognition is an important research area in...
research
03/16/2018

Decision support with text-based emotion recognition: Deep learning for affective computing

Emotions widely affect the decision-making of humans and, hence, affecti...
research
06/03/2021

"You made me feel this way": Investigating Partners' Influence in Predicting Emotions in Couples' Conflict Interactions using Speech Data

How romantic partners interact with each other during a conflict influen...
research
05/30/2011

Neural Networks for Emotion Classification

It is argued that for the computer to be able to interact with humans, i...

Please sign up or login with your details

Forgot password? Click here to reset