HIBRIDS: Attention with Hierarchical Biases for Structure-aware Long Document Summarization

03/21/2022
by   Shuyang Cao, et al.
0

Document structure is critical for efficient information consumption. However, it is challenging to encode it efficiently into the modern Transformer architecture. In this work, we present HIBRIDS, which injects Hierarchical Biases foR Incorporating Document Structure into the calculation of attention scores. We further present a new task, hierarchical question-summary generation, for summarizing salient content in the source document into a hierarchy of questions and summaries, where each follow-up question inquires about the content of its parent question-summary pair. We also annotate a new dataset with 6,153 question-summary hierarchies labeled on long government reports. Experiment results show that our model produces better question-summary hierarchies than comparisons on both hierarchy quality and content coverage, a finding also echoed by human judges. Additionally, our model improves the generation of long-form summaries from lengthy government reports and Wikipedia articles, as measured by ROUGE scores.

READ FULL TEXT
research
05/14/2019

Ontology-Aware Clinical Abstractive Summarization

Automatically generating accurate summaries from clinical reports could ...
research
08/28/2022

Podcast Summary Assessment: A Resource for Evaluating Summary Assessment Methods

Automatic summary assessment is useful for both machine-generated and hu...
research
11/15/2021

Question-Based Salient Span Selection for More Controllable Text Summarization

In this work, we propose a method for incorporating question-answering (...
research
05/24/2023

AWESOME: GPU Memory-constrained Long Document Summarization using Memory Mechanism and Global Salient Content

Long document summarization systems are critical for domains with length...
research
05/25/2022

Factorizing Content and Budget Decisions in Abstractive Summarization of Long Documents by Sampling Summary Views

We argue that disentangling content selection from the budget used to co...
research
11/17/2022

Abstractive Summarization Guided by Latent Hierarchical Document Structure

Sequential abstractive neural summarizers often do not use the underlyin...
research
10/22/2022

ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts

Despite tremendous progress in automatic summarization, state-of-the-art...

Please sign up or login with your details

Forgot password? Click here to reset