Investigation of Multimodal Features, Classifiers and Fusion Methods for Emotion Recognition

by   Zheng Lian, et al.

Automatic emotion recognition is a challenging task. In this paper, we present our effort for the audio-video based sub-challenge of the Emotion Recognition in the Wild (EmotiW) 2018 challenge, which requires participants to assign a single emotion label to the video clip from the six universal emotions (Anger, Disgust, Fear, Happiness, Sad and Surprise) and Neutral. The proposed multimodal emotion recognition system takes audio, video and text information into account. Except for handcraft features, we also extract bottleneck features from deep neutral networks (DNNs) via transfer learning. Both temporal classifiers and non-temporal classifiers are evaluated to obtain the best unimodal emotion classification result. Then possibilities are extracted and passed into the Beam Search Fusion (BS-Fusion). We test our method in the EmotiW 2018 challenge and we gain promising results. Compared with the baseline system, there is a significant improvement. We achieve 60.34 testing dataset, which is only 1.5 method is very competitive.



There are no comments yet.


page 1

page 2

page 3

page 4


Analyzing the Influence of Dataset Composition for Emotion Recognition

Recognizing emotions from text in multimodal architectures has yielded p...

Multimodal Emotion Recognition for One-Minute-Gradual Emotion Challenge

The continuous dimensional emotion modelled by arousal and valence can d...

Spatiotemporal Networks for Video Emotion Recognition

Our experiment adapts several popular deep learning methods as well as s...

Feature-level and Model-level Audiovisual Fusion for Emotion Recognition in the Wild

Emotion recognition plays an important role in human-computer interactio...

EmoNets: Multimodal deep learning approaches for emotion recognition in video

The task of the emotion recognition in the wild (EmotiW) Challenge is to...

Temporal Multimodal Fusion for Video Emotion Classification in the Wild

This paper addresses the question of emotion classification. The task co...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.