Discourse-Aware Neural Rewards for Coherent Text Generation

05/10/2018
by   Antoine Bosselut, et al.
0

In this paper, we investigate the use of discourse-aware rewards with reinforcement learning to guide a model to generate long, coherent text. In particular, we propose to learn neural rewards to model cross-sentence ordering as a means to approximate desired discourse structure. Empirical results demonstrate that a generator trained with the learned reward produces more coherent and less repetitive text than models trained with cross-entropy or with reinforcement learning with commonly used scores as rewards.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/14/2018

Modeling Coherence for Discourse Neural Machine Translation

Discourse coherence plays an important role in the translation of one te...
research
02/01/2019

Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models

Variational autoencoders (VAEs) have received much attention recently as...
research
10/12/2021

DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer

Despite the recent advances in applying pre-trained language models to g...
research
12/10/2021

Discourse-Aware Prompt Design for Text Generation

Current efficient fine-tuning methods (e.g., adapters, prefix-tuning, et...
research
04/30/2018

Towards Diverse Text Generation with Inverse Reinforcement Learning

Text generation is a crucial task in NLP. Recently, several adversarial ...
research
08/30/2019

Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs

Generating a long, coherent text such as a paragraph requires a high-lev...
research
10/14/2020

Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries

Long-text generation remains a challenge. The difficulty of generating c...

Please sign up or login with your details

Forgot password? Click here to reset