Capturing Global Structural Information in Long Document Question Answering with Compressive Graph Selector Network

10/11/2022
by   Yuxiang Nie, et al.
0

Long document question answering is a challenging task due to its demands for complex reasoning over long text. Previous works usually take long documents as non-structured flat texts or only consider the local structure in long documents. However, these methods usually ignore the global structure of the long document, which is essential for long-range understanding. To tackle this problem, we propose Compressive Graph Selector Network (CGSN) to capture the global structure in a compressive and iterative manner. Specifically, the proposed model consists of three modules: local graph network, global graph network and evidence memory network. Firstly, the local graph network builds the graph structure of the chunked segment in token, sentence, paragraph and segment levels to capture the short-term dependency of the text. Secondly, the global graph network selectively receives the information of each level from the local graph, compresses them into the global graph nodes and applies graph attention into the global graph nodes to build the long-range reasoning over the entire text in an iterative way. Thirdly, the evidence memory network is designed to alleviate the redundancy problem in the evidence selection via saving the selected result in the previous steps. Extensive experiments show that the proposed model outperforms previous methods on two datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/03/2023

AttenWalker: Unsupervised Long-Document Question Answering via Attention-based Graph Walking

Annotating long-document question answering (long-document QA) pairs is ...
research
12/16/2021

Utilizing Evidence Spans via Sequence-Level Contrastive Learning for Long-Context Question Answering

Long-range transformer models have achieved encouraging results on long-...
research
03/30/2022

Graph Refinement for Coreference Resolution

The state-of-the-art models for coreference resolution are based on inde...
research
05/10/2021

ReadTwice: Reading Very Large Documents with Memories

Knowledge-intensive tasks such as question answering often require assim...
research
08/22/2023

Knowledge Graph Prompting for Multi-Document Question Answering

The 'pre-train, prompt, predict' paradigm of large language models (LLMs...
research
11/20/2022

Semi-supervised Local Cluster Extraction by Compressive Sensing

Local clustering problem aims at extracting a small local structure insi...
research
02/22/2022

Socialformer: Social Network Inspired Long Document Modeling for Document Ranking

Utilizing pre-trained language models has achieved great success for neu...

Please sign up or login with your details

Forgot password? Click here to reset