MEG: Multi-Evidence GNN for Multimodal Semantic Forensics

11/23/2020
by   Ekraam Sabir, et al.
3

Fake news often involves semantic manipulations across modalities such as image, text, location etc and requires the development of multimodal semantic forensics for its detection. Recent research has centered the problem around images, calling it image repurposing – where a digitally unmanipulated image is semantically misrepresented by means of its accompanying multimodal metadata such as captions, location, etc. The image and metadata together comprise a multimedia package. The problem setup requires algorithms to perform multimodal semantic forensics to authenticate a query multimedia package using a reference dataset of potentially related packages as evidences. Existing methods are limited to using a single evidence (retrieved package), which ignores potential performance improvement from the use of multiple evidences. In this work, we introduce a novel graph neural network based model for multimodal semantic forensics, which effectively utilizes multiple retrieved packages as evidences and is scalable with the number of evidences. We compare the scalability and performance of our model against existing methods. Experimental results show that the proposed model outperforms existing state-of-the-art algorithms with an error reduction of up to 25

READ FULL TEXT

page 1

page 6

page 7

page 8

research
07/06/2017

Multimedia Semantic Integrity Assessment Using Joint Embedding Of Images And Text

Real world multimedia data is often composed of multiple modalities such...
research
02/25/2022

GAME-ON: Graph Attention Network based Multimodal Fusion for Fake News Detection

Social media in present times has a significant and growing influence. F...
research
08/20/2018

Deep Multimodal Image-Repurposing Detection

Nefarious actors on social media and other platforms often spread rumors...
research
12/12/2022

Multimodal Matching-aware Co-attention Networks with Mutual Knowledge Distillation for Fake News Detection

Fake news often involves multimedia information such as text and image t...
research
11/15/2022

MM-Locate-News: Multimodal Focus Location Estimation in News

The consumption of news has changed significantly as the Web has become ...
research
06/15/2023

Training Multimedia Event Extraction With Generated Images and Captions

Contemporary news reporting increasingly features multimedia content, mo...
research
03/02/2019

AIRD: Adversarial Learning Framework for Image Repurposing Detection

Image repurposing is a commonly used method for spreading misinformation...

Please sign up or login with your details

Forgot password? Click here to reset