DeepAI AI Chat
Log In Sign Up

Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response

04/14/2020
by   Ferda Ofli, et al.
Hamad Bin Khalifa University
0

Multimedia content in social media platforms provides significant information during disaster events. The types of information shared include reports of injured or deceased people, infrastructure damage, and missing or found people, among others. Although many studies have shown the usefulness of both text and image content for disaster response purposes, the research has been mostly focused on analyzing only the text modality in the past. In this paper, we propose to use both text and image modalities of social media data to learn a joint representation using state-of-the-art deep learning techniques. Specifically, we utilize convolutional neural networks to define a multimodal deep learning architecture with a modality-agnostic shared representation. Extensive experiments on real-world disaster datasets show that the proposed multimodal architecture yields better performance than models trained using a single modality (e.g., either text or image).

READ FULL TEXT
10/18/2022

MMGA: Multimodal Learning with Graph Alignment

Multimodal pre-training breaks down the modality barriers and allows the...
09/23/2021

MARMOT: A Deep Learning Framework for Constructing Multimodal Representations for Vision-and-Language Tasks

Political activity on social media presents a data-rich window into poli...
09/01/2019

Employ Multimodal Machine Learning for Content quality analysis

The task of identifying high-quality content becomes increasingly import...
05/02/2018

CrisisMMD: Multimodal Twitter Datasets from Natural Disasters

During natural and man-made disasters, people use social media platforms...
04/10/2020

Multimodal Categorization of Crisis Events in Social Media

Recent developments in image classification and natural language process...
05/17/2023

Rethinking Multimodal Content Moderation from an Asymmetric Angle with Mixed-modality

There is a rapidly growing need for multimodal content moderation (CM) a...
04/27/2023

Figments and Misalignments: A Framework for Fine-grained Crossmodal Misinformation Detection

Multimedia content has become ubiquitous on social media platforms, lead...