Stimuli-Aware Visual Emotion Analysis

09/04/2021
by   Jingyuan Yang, et al.
5

Visual emotion analysis (VEA) has attracted great attention recently, due to the increasing tendency of expressing and understanding emotions through images on social networks. Different from traditional vision tasks, VEA is inherently more challenging since it involves a much higher level of complexity and ambiguity in human cognitive process. Most of the existing methods adopt deep learning techniques to extract general features from the whole image, disregarding the specific features evoked by various emotional stimuli. Inspired by the Stimuli-Organism-Response (S-O-R) emotion model in psychological theory, we proposed a stimuli-aware VEA method consisting of three stages, namely stimuli selection (S), feature extraction (O) and emotion prediction (R). First, specific emotional stimuli (i.e., color, object, face) are selected from images by employing the off-the-shelf tools. To the best of our knowledge, it is the first time to introduce stimuli selection process into VEA in an end-to-end network. Then, we design three specific networks, i.e., Global-Net, Semantic-Net and Expression-Net, to extract distinct emotional features from different stimuli simultaneously. Finally, benefiting from the inherent structure of Mikel's wheel, we design a novel hierarchical cross-entropy loss to distinguish hard false examples from easy ones in an emotion-specific manner. Experiments demonstrate that the proposed method consistently outperforms the state-of-the-art approaches on four public visual emotion datasets. Ablation study and visualizations further prove the validity and interpretability of our method.

READ FULL TEXT

page 1

page 2

page 5

page 6

page 10

page 11

page 12

page 14

research
10/24/2021

SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network

Visual Emotion Analysis (VEA) aims at finding out how people feel emotio...
research
08/29/2022

From Smart Sensing to Consciousness: An info-structural model of computational consciousness for non-interacting agents

This study proposes a model of computational consciousness for non-inter...
research
06/23/2021

A Circular-Structured Representation for Visual Emotion Distribution Learning

Visual Emotion Analysis (VEA) has attracted increasing attention recentl...
research
07/25/2022

Seeking Subjectivity in Visual Emotion Distribution Learning

Visual Emotion Analysis (VEA), which aims to predict people's emotions t...
research
11/22/2020

Investigating Emotion-Color Association in Deep Neural Networks

It has been found that representations learned by Deep Neural Networks (...
research
03/23/2020

Information-Theoretic Free Energy as Emotion Potential: Emotional Valence as a Function of Complexity and Novelty

This study extends the mathematical model of emotion dimensions that we ...
research
07/16/2023

EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributes

Visual Emotion Analysis (VEA) aims at predicting people's emotional resp...

Please sign up or login with your details

Forgot password? Click here to reset