EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle

03/14/2020
by   Trisha Mittal, et al.
24

We present EmotiCon, a learning-based algorithm for context-aware perceived human emotion recognition from videos and images. Motivated by Frege's Context Principle from psychology, our approach combines three interpretations of context for emotion recognition. Our first interpretation is based on using multiple modalities(e.g. faces and gaits) for emotion recognition. For the second interpretation, we gather semantic context from the input image and use a self-attention-based CNN to encode this information. Finally, we use depth maps to model the third interpretation related to socio-dynamic interactions and proximity among agents. We demonstrate the efficiency of our network through experiments on EMOTIC, a benchmark dataset. We report an Average Precision (AP) score of 35.48 across 26 classes, which is an improvement of 7-8 over prior methods. We also introduce a new dataset, GroupWalk, which is a collection of videos captured in multiple real-world settings of people walking. We report an AP of 65.83 across 4 categories on GroupWalk, which is also an improvement over prior methods.

READ FULL TEXT

page 1

page 3

page 7

page 8

page 11

page 12

research
08/16/2019

Context-Aware Emotion Recognition Networks

Traditional techniques for emotion recognition have focused on the facia...
research
07/06/2023

SeLiNet: Sentiment enriched Lightweight Network for Emotion Recognition in Images

In this paper, we propose a sentiment-enriched lightweight network SeLiN...
research
11/09/2019

M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues

We present M3ER, a learning-based method for emotion recognition from mu...
research
05/03/2018

Transformer for Emotion Recognition

This paper describes the UMONS solution for the OMG-Emotion Challenge. W...
research
08/25/2022

Interpretable Multimodal Emotion Recognition using Hybrid Fusion of Speech and Image Data

This paper proposes a multimodal emotion recognition system based on hyb...
research
02/02/2022

Interpretability for Multimodal Emotion Recognition using Concept Activation Vectors

Multimodal Emotion Recognition refers to the classification of input vid...
research
11/30/2021

Affect-DML: Context-Aware One-Shot Recognition of Human Affect using Deep Metric Learning

Human affect recognition is a well-established research area with numero...

Please sign up or login with your details

Forgot password? Click here to reset