When Saliency Meets Sentiment: Understanding How Image Content Invokes Emotion and Sentiment

by   Honglin Zheng, et al.

Sentiment analysis is crucial for extracting social signals from social media content. Due to the prevalence of images in social media, image sentiment analysis is receiving increasing attention in recent years. However, most existing systems are black-boxes that do not provide insight on how image content invokes sentiment and emotion in the viewers. Psychological studies have confirmed that salient objects in an image often invoke emotions. In this work, we investigate more fine-grained and more comprehensive interaction between visual saliency and visual sentiment. In particular, we partition images in several primary scene-type dimensions, including: open-closed, natural-manmade, indoor-outdoor, and face-noface. Using state of the art saliency detection algorithm and sentiment classification algorithm, we examine how the sentiment of the salient region(s) in an image relates to the overall sentiment of the image. The experiments on a representative image emotion dataset have shown interesting correlation between saliency and sentiment in different scene types and in turn shed light on the mechanism of visual sentiment evocation.



page 1

page 3


Sentiment Analysis from Images of Natural Disasters

Social media have been widely exploited to detect and gather relevant in...

An AutoML-based Approach to Multimodal Image Sentiment Analysis

Sentiment analysis is a research topic focused on analysing data to extr...

From Pixels to Sentiment: Fine-tuning CNNs for Visual Sentiment Prediction

Visual multimedia have become an inseparable part of our digital social ...

Multimodal Dual Emotion with Fusion of Visual Sentiment for Rumor Detection

In recent years, rumors have had a devastating impact on society, making...

Smile, be Happy :) Emoji Embedding for Visual Sentiment Analysis

Due to the lack of large-scale datasets, the prevailing approach in visu...

Multi-task Pairwise Neural Ranking for Hashtag Segmentation

Hashtags are often employed on social media and beyond to add metadata t...

OutdoorSent: Sentiment Analysis of Urban Outdoor Images by Using Semantic and Deep Features

Opinion mining in outdoor images posted by users during day-to-day or le...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.