Focused Attention Improves Document-Grounded Generation

04/26/2021
by   Shrimai Prabhumoye, et al.
26

Document grounded generation is the task of using the information provided in a document to improve text generation. This work focuses on two different document grounded generation tasks: Wikipedia Update Generation task and Dialogue response generation. Our work introduces two novel adaptations of large scale pre-trained encoder-decoder models focusing on building context driven representation of the document and enabling specific attention to the information in the document. Additionally, we provide a stronger BART baseline for these tasks. Our proposed techniques outperform existing methods on both automated (at least 48 closeness to reference and relevance to the document. Furthermore, we perform comprehensive manual inspection of the generated output and categorize errors to provide insights into future directions in modeling these tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/14/2021

Joint Retrieval and Generation Training for Grounded Text Generation

Recent advances in large-scale pre-training such as GPT-3 allow seemingl...
research
12/02/2021

KPDrop: An Approach to Improving Absent Keyphrase Generation

Keyphrase generation is the task of generating phrases (keyphrases) that...
research
04/16/2022

UniGDD: A Unified Generative Framework for Goal-Oriented Document-Grounded Dialogue

The goal-oriented document-grounded dialogue aims at responding to the u...
research
10/14/2021

Hindsight: Posterior-guided training of retrievers for improved open-ended generation

Many text generation systems benefit from using a retriever to retrieve ...
research
09/10/2021

Heterogeneous Graph Neural Networks for Keyphrase Generation

The encoder-decoder framework achieves state-of-the-art results in keyph...
research
10/20/2020

Elaborative Simplification: Content Addition and Explanation Generation in Text Simplification

Much of modern day text simplification research focuses on sentence-leve...
research
10/31/2022

Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model

In this work, we present a model for document-grounded response generati...

Please sign up or login with your details

Forgot password? Click here to reset