Log In Sign Up

Late Fusion with Triplet Margin Objective for Multimodal Ideology Prediction and Analysis

by   Changyuan Qiu, et al.

Prior work on ideology prediction has largely focused on single modalities, i.e., text or images. In this work, we introduce the task of multimodal ideology prediction, where a model predicts binary or five-point scale ideological leanings, given a text-image pair with political content. We first collect five new large-scale datasets with English documents and images along with their ideological leanings, covering news articles from a wide range of US mainstream media and social media posts from Reddit and Twitter. We conduct in-depth analyses of news articles and reveal differences in image content and usage across the political spectrum. Furthermore, we perform extensive experiments and ablation studies, demonstrating the effectiveness of targeted pretraining objectives on different model components. Our best-performing model, a late-fusion architecture pretrained with a triplet objective over multimodal content, outperforms the state-of-the-art text-only model by almost 4


page 1

page 5


POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection

Ideology is at the core of political science research. Yet, there still ...

QuTI! Quantifying Text-Image Consistency in Multimodal Documents

The World Wide Web and social media platforms have become popular source...

We Can Detect Your Bias: Predicting the Political Ideology of News Articles

We explore the task of predicting the leading political ideology or bias...

MARMOT: A Deep Learning Framework for Constructing Multimodal Representations for Vision-and-Language Tasks

Political activity on social media presents a data-rich window into poli...

Multimodal Emoji Prediction

Emojis are small images that are commonly included in social media text ...

Story-oriented Image Selection and Placement

Multimodal contents have become commonplace on the Internet today, manif...

Employ Multimodal Machine Learning for Content quality analysis

The task of identifying high-quality content becomes increasingly import...