DeepAI
Log In Sign Up

Focused Attention Improves Document-Grounded Generation

04/26/2021
by   Shrimai Prabhumoye, et al.
26

Document grounded generation is the task of using the information provided in a document to improve text generation. This work focuses on two different document grounded generation tasks: Wikipedia Update Generation task and Dialogue response generation. Our work introduces two novel adaptations of large scale pre-trained encoder-decoder models focusing on building context driven representation of the document and enabling specific attention to the information in the document. Additionally, we provide a stronger BART baseline for these tasks. Our proposed techniques outperform existing methods on both automated (at least 48 closeness to reference and relevance to the document. Furthermore, we perform comprehensive manual inspection of the generated output and categorize errors to provide insights into future directions in modeling these tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/14/2021

Joint Retrieval and Generation Training for Grounded Text Generation

Recent advances in large-scale pre-training such as GPT-3 allow seemingl...
12/02/2021

KPDrop: An Approach to Improving Absent Keyphrase Generation

Keyphrase generation is the task of generating phrases (keyphrases) that...
07/20/2019

Incremental Transformer with Deliberation Decoder for Document Grounded Conversations

Document Grounded Conversations is a task to generate dialogue responses...
10/14/2021

Hindsight: Posterior-guided training of retrievers for improved open-ended generation

Many text generation systems benefit from using a retriever to retrieve ...
09/10/2021

Heterogeneous Graph Neural Networks for Keyphrase Generation

The encoder-decoder framework achieves state-of-the-art results in keyph...
10/20/2020

Elaborative Simplification: Content Addition and Explanation Generation in Text Simplification

Much of modern day text simplification research focuses on sentence-leve...
06/14/2021

Cascaded Span Extraction and Response Generation for Document-Grounded Dialog

This paper summarizes our entries to both subtasks of the first DialDoc ...

Code Repositories

Focused-Attention-Improves-Document-Grounded-Generation

None


view repo