Is Graph Structure Necessary for Multi-hop Reasoning?

04/07/2020
by   Nan Shao, et al.
0

Recently, many works attempt to model texts as graph structure and introduce graph neural networks to deal with it on many NLP tasks.In this paper, we investigate whether graph structure is necessary for multi-hop reasoning tasks and what role it plays. Our analysis is centered on HotpotQA. We use the state-of-the-art published model, Dynamically Fused Graph Network (DFGN), as our baseline. By directly modifying the pre-trained model, our baseline model gains a large improvement and significantly surpass both published and unpublished works. Ablation experiments established that, with the proper use of pre-trained models, graph structure may not be necessary for multi-hop reasoning. We point out that both the graph structure and the adjacency matrix are task-related prior knowledge, and graph-attention can be considered as a special case of self-attention. Experiments demonstrate that graph-attention or the entire graph structure can be replaced by self-attention or Transformers, and achieve similar results to the previous state-of-the-art model achieved.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/29/2020

Direct Multi-hop Attention based Graph Neural Network

Introducing self-attention mechanism in graph neural networks (GNNs) ach...
research
11/12/2022

Self-Supervised Graph Structure Refinement for Graph Neural Networks

Graph structure learning (GSL), which aims to learn the adjacency matrix...
research
07/25/2021

Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy

Multi-hop reading comprehension (MHRC) requires not only to predict the ...
research
04/21/2023

Self-Attention in Colors: Another Take on Encoding Graph Structure in Transformers

We introduce a novel self-attention mechanism, which we call CSA (Chroma...
research
08/15/2023

Enhancing Visually-Rich Document Understanding via Layout Structure Modeling

In recent years, the use of multi-modal pre-trained Transformers has led...
research
07/01/2023

Single Sequence Prediction over Reasoning Graphs for Multi-hop QA

Recent generative approaches for multi-hop question answering (QA) utili...
research
06/15/2012

Improving the Asymmetric TSP by Considering Graph Structure

Recent works on cost based relaxations have improved Constraint Programm...

Please sign up or login with your details

Forgot password? Click here to reset