DeepAI AI Chat
Log In Sign Up

Which Emoji Talks Best for My Picture?

by   Anurag Illendula, et al.
IIT Kharagpur

Emojis have evolved as complementary sources for expressing emotion in social-media platforms where posts are mostly composed of texts and images. In order to increase the expressiveness of the social media posts, users associate relevant emojis with their posts. Incorporating domain knowledge has improved machine understanding of text. In this paper, we investigate whether domain knowledge for emoji can improve the accuracy of emoji recommendation task in case of multimedia posts composed of image and text. Our emoji recommendation can suggest accurate emojis by exploiting both visual and textual content from social media posts as well as domain knowledge from Emojinet. Experimental results using pre-trained image classifiers and pre-trained word embedding models on Twitter dataset show that our results outperform the current state-of-the-art by 9.6%. We also present a user study evaluation of our recommendation system on a set of images chosen from MSCOCO dataset.


page 1

page 2


Multimodal Emotion Classification

Most NLP and Computer Vision tasks are limited to scarcity of labelled d...

An Empirical Evaluation of Text Representation Schemes on Multilingual Social Web to Filter the Textual Aggression

This paper attempt to study the effectiveness of text representation sch...

Portuguese Man-of-War Image Classification with Convolutional Neural Networks

Portuguese man-of-war (PMW) is a gelatinous organism with long tentacles...

DataSite: Proactive Visual Data Exploration with Computation of Insight-based Recommendations

Effective data analysis ideally requires the analyst to have high expert...

Deep Text Mining of Instagram Data Without Strong Supervision

With the advent of social media, our online feeds increasingly consist o...

Multimodal Emoji Prediction

Emojis are small images that are commonly included in social media text ...

Understanding and Detecting Hateful Content using Contrastive Learning

The spread of hate speech and hateful imagery on the Web is a significan...