DeepAI AI Chat
Log In Sign Up

Point-of-Interest Type Prediction using Text and Images

by   Danae Sanchez Villegas, et al.
The University of Sheffield

Point-of-interest (POI) type prediction is the task of inferring the type of a place from where a social media post was shared. Inferring a POI's type is useful for studies in computational social science including sociolinguistics, geosemiotics, and cultural geography, and has applications in geosocial networking technologies such as recommendation and visualization systems. Prior efforts in POI type prediction focus solely on text, without taking visual information into account. However in reality, the variety of modalities, as well as their semiotic relationships with one another, shape communication and interactions in social media. This paper presents a study on POI type prediction using multimodal information from text and images available at posting time. For that purpose, we enrich a currently available data set for POI type prediction with the images that accompany the text messages. Our proposed method extracts relevant information from each modality to effectively capture interactions between text and image achieving a macro F1 of 47.21 across eight categories significantly outperforming the state-of-the-art method for POI type prediction based on text-only methods. Finally, we provide a detailed analysis to shed light on cross-modal interactions and the limitations of our best performing model.


page 1

page 8

page 9


Point-of-Interest Type Inference from Social Media Text

Physical places help shape how we perceive the experiences we have there...

MemeTector: Enforcing deep focus for meme detection

Image memes and specifically their widely-known variation image macros, ...

The Emotions of the Crowd: Learning Image Sentiment from Tweets via Cross-modal Distillation

Trends and opinion mining in social media increasingly focus on novel in...

Cross-Media Keyphrase Prediction: A Unified Framework with Multi-Modality Multi-Head Attention and Image Wordings

Social media produces large amounts of contents every day. To help users...

Exploring Emoji Usage and Prediction Through a Temporal Variation Lens

The frequent use of Emojis on social media platforms has created a new f...

Multimodal Emoji Prediction

Emojis are small images that are commonly included in social media text ...

Understanding eWhoring

In this paper, we describe a new type of online fraud, referred to as 'e...