Multimodal Analogical Reasoning over Knowledge Graphs

10/01/2022
by   Ningyu Zhang, et al.
0

Analogical reasoning is fundamental to human cognition and holds an important place in various fields. However, previous studies mainly focus on single-modal analogical reasoning and ignore taking advantage of structure knowledge. Notably, the research in cognitive psychology has demonstrated that information from multimodal sources always brings more powerful cognitive transfer than single modality sources. To this end, we introduce the new task of multimodal analogical reasoning over knowledge graphs, which requires multimodal reasoning ability with the help of background knowledge. Specifically, we construct a Multimodal Analogical Reasoning dataSet (MARS) and a multimodal knowledge graph MarKG. We evaluate with multimodal knowledge graph embedding and pre-trained Transformer baselines, illustrating the potential challenges of the proposed task. We further propose a novel model-agnostic Multimodal analogical reasoning framework with Transformer (MarT) motivated by the structure mapping theory, which can obtain better performance. Code and datasets are available in https://github.com/zjunlp/MKG_Analogy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/15/2022

Knowledge Graph Completion with Pre-trained Multimodal Transformer and Twins Negative Sampling

Knowledge graphs (KGs) that modelings the world knowledge as structural ...
research
06/13/2019

Cognitive Knowledge Graph Reasoning for One-shot Relational Learning

Inferring new facts from existing knowledge graphs (KG) with explainable...
research
01/24/2022

A Knowledge Graph Embeddings based Approach for Author Name Disambiguation using Literals

Scholarly data is growing continuously containing information about the ...
research
04/05/2023

Personality-aware Human-centric Multimodal Reasoning: A New Task

Multimodal reasoning, an area of artificial intelligence that aims at ma...
research
06/18/2022

GaLeNet: Multimodal Learning for Disaster Prediction, Management and Relief

After a natural disaster, such as a hurricane, millions are left in need...
research
05/04/2022

Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion

Multimodal Knowledge Graphs (MKGs), which organize visual-text factual k...
research
04/06/2021

Text-guided Legal Knowledge Graph Reasoning

Recent years have witnessed the prosperity of legal artificial intellige...

Please sign up or login with your details

Forgot password? Click here to reset