RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarization

by   Chenxin An, et al.
University of Illinois at Urbana-Champaign
FUDAN University

Existing summarization systems mostly generate summaries purely relying on the content of the source document. However, even for humans, we usually need some references or exemplars to help us fully understand the source document and write summaries in a particular format. But how to find the high-quality exemplars and incorporate them into summarization systems is still challenging and worth exploring. In this paper, we propose RetrievalSum, a novel retrieval enhanced abstractive summarization framework consisting of a dense Retriever and a Summarizer. At first, several closely related exemplars are retrieved as supplementary input to help the generation model understand the text more comprehensively. Furthermore, retrieved exemplars can also play a role in guiding the model to capture the writing style of a specific corpus. We validate our method on a wide range of summarization datasets across multiple domains and two backbone models: BERT and BART. Results show that our framework obtains significant improvement by 1.38 4.66 in ROUGE-1 score when compared with the powerful pre-trained models, and achieve new state-of-the-art on BillSum. Human evaluation demonstrates that our retrieval enhanced model can better capture the domain-specific writing style.


page 1

page 2

page 3

page 4


How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization?

Automatic summarization of legal case judgements has traditionally been ...

Towards Improving Faithfulness in Abstractive Summarization

Despite the success achieved in neural abstractive summarization based o...

Domain Adaptation with Pre-trained Transformers for Query Focused Abstractive Text Summarization

The Query Focused Text Summarization (QFTS) task aims at building system...

Hooks in the Headline: Learning to Generate Headlines with Controlled Styles

Current summarization systems only produce plain, factual headlines, but...

SQuALITY: Building a Long-Document Summarization Dataset the Hard Way

Summarization datasets are often assembled either by scraping naturally ...

Exploiting local and global performance of candidate systems for aggregation of summarization techniques

With an ever growing number of extractive summarization techniques being...

QontSum: On Contrasting Salient Content for Query-focused Summarization

Query-focused summarization (QFS) is a challenging task in natural langu...

Please sign up or login with your details

Forgot password? Click here to reset