DeepAI AI Chat
Log In Sign Up

QuTI! Quantifying Text-Image Consistency in Multimodal Documents

by   Matthias Springstein, et al.

The World Wide Web and social media platforms have become popular sources for news and information. Typically, multimodal information, e.g., image and text is used to convey information more effectively and to attract attention. While in most cases image content is decorative or depicts additional information, it has also been leveraged to spread misinformation and rumors in recent years. In this paper, we present a Web-based demo application that automatically quantifies the cross-modal relations of entities (persons, locations, and events) in image and text. The applications are manifold. For example, the system can help users to explore multimodal articles more efficiently, or can assist human assessors and fact-checking efforts in the verification of the credibility of news stories, tweets, or other multimodal documents.


Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency

The World Wide Web has become a popular source for gathering information...

Preserving Semantic Neighborhoods for Robust Cross-modal Retrieval

The abundance of multimodal data (e.g. social media posts) has inspired ...

Understanding, Categorizing and Predicting Semantic Image-Text Relations

Two modalities are often used to convey information in a complementary a...

Late Fusion with Triplet Margin Objective for Multimodal Ideology Prediction and Analysis

Prior work on ideology prediction has largely focused on single modaliti...

A Modality-level Explainable Framework for Misinformation Checking in Social Networks

The widespread of false information is a rising concern worldwide with c...

Story-oriented Image Selection and Placement

Multimodal contents have become commonplace on the Internet today, manif...

Synthetic Misinformers: Generating and Combating Multimodal Misinformation

With the expansion of social media and the increasing dissemination of m...