Log In Sign Up

A Generalized Zero-Shot Framework for Emotion Recognition from Body Gestures

by   Jinting Wu, et al.

Although automatic emotion recognition from facial expressions and speech has made remarkable progress, emotion recognition from body gestures has not been thoroughly explored. People often use a variety of body language to express emotions, and it is difficult to enumerate all emotional body gestures and collect enough samples for each category. Therefore, recognizing new emotional body gestures is critical for better understanding human emotions. However, the existing methods fail to accurately determine which emotional state a new body gesture belongs to. In order to solve this problem, we introduce a Generalized Zero-Shot Learning (GZSL) framework, which consists of three branches to infer the emotional state of the new body gestures with only their semantic descriptions. The first branch is a Prototype-Based Detector (PBD) which is used to determine whether an sample belongs to a seen body gesture category and obtain the prediction results of the samples from the seen categories. The second branch is a Stacked AutoEncoder (StAE) with manifold regularization, which utilizes semantic representations to predict samples from unseen categories. Note that both of the above branches are for body gesture recognition. We further add an emotion classifier with a softmax layer as the third branch in order to better learn the feature representations for this emotion classification task. The input features for these three branches are learned by a shared feature extraction network, i.e., a Bidirectional Long Short-Term Memory Networks (BLSTM) with a self-attention module. We treat these three branches as subtasks and use multi-task learning strategies for joint training. The performance of our framework on an emotion recognition dataset is significantly superior to the traditional method of emotion classification and state-of-the-art zero-shot learning methods.


page 1

page 8


A Prototype-Based Generalized Zero-Shot Learning Framework for Hand Gesture Recognition

Hand gesture recognition plays a significant role in human-computer inte...

Survey on Emotional Body Gesture Recognition

Automatic emotion recognition has become a trending research topic in th...

iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis

We introduce a new dataset for the emotional artificial intelligence res...

Deformable Convolutional LSTM for Human Body Emotion Recognition

People represent their emotions in a myriad of ways. Among the most impo...

Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation

Inspired by the human ability to infer emotions from body language, we p...

Vision based body gesture meta features for Affective Computing

Early detection of psychological distress is key to effective treatment....