Focused Attention Improves Document-Grounded Generation

by   Shrimai Prabhumoye, et al.

Document grounded generation is the task of using the information provided in a document to improve text generation. This work focuses on two different document grounded generation tasks: Wikipedia Update Generation task and Dialogue response generation. Our work introduces two novel adaptations of large scale pre-trained encoder-decoder models focusing on building context driven representation of the document and enabling specific attention to the information in the document. Additionally, we provide a stronger BART baseline for these tasks. Our proposed techniques outperform existing methods on both automated (at least 48 closeness to reference and relevance to the document. Furthermore, we perform comprehensive manual inspection of the generated output and categorize errors to provide insights into future directions in modeling these tasks.


page 1

page 2

page 3

page 4


Joint Retrieval and Generation Training for Grounded Text Generation

Recent advances in large-scale pre-training such as GPT-3 allow seemingl...

KPDrop: An Approach to Improving Absent Keyphrase Generation

Keyphrase generation is the task of generating phrases (keyphrases) that...

UniGDD: A Unified Generative Framework for Goal-Oriented Document-Grounded Dialogue

The goal-oriented document-grounded dialogue aims at responding to the u...

Hindsight: Posterior-guided training of retrievers for improved open-ended generation

Many text generation systems benefit from using a retriever to retrieve ...

Heterogeneous Graph Neural Networks for Keyphrase Generation

The encoder-decoder framework achieves state-of-the-art results in keyph...

Elaborative Simplification: Content Addition and Explanation Generation in Text Simplification

Much of modern day text simplification research focuses on sentence-leve...

Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model

In this work, we present a model for document-grounded response generati...

Code Repositories



view repo

Please sign up or login with your details

Forgot password? Click here to reset