DeepAI AI Chat
Log In Sign Up

RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators

by   Rilwan A. Adewoyin, et al.

In this paper, we study the task of improving the cohesion and coherence of long-form text generated by language models. To this end, we propose RSTGen, a framework that utilises Rhetorical Structure Theory (RST), a classical language theory, to control the discourse structure, semantics and topics of generated text. Firstly, we demonstrate our model's ability to control structural discourse and semantic features of generated text in open generation evaluation. Then we experiment on the two challenging long-form text tasks of argument generation and story generation. Evaluation using automated metrics and a metric with high correlation to human evaluation, shows that our model performs competitively against existing models, while offering significantly more controls over generated text than alternative methods.


page 1

page 2

page 3

page 4


Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence

Generating long and coherent text is an important but challenging task, ...

FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation

Evaluating the factuality of long-form text generated by large language ...

Plug-and-Blend: A Framework for Controllable Story Generation with Blended Control Codes

We describe a Plug-and-Play controllable language generation framework, ...

Modeling Topical Coherence in Discourse without Supervision

Coherence of text is an important attribute to be measured for both manu...

Strategies for Structuring Story Generation

Writers generally rely on plans or sketches to write long stories, but m...

Cluster-based Evaluation of Automatically Generated Text

While probabilistic language generators have improved dramatically over ...

Leveraging Natural Supervision for Language Representation Learning and Generation

Recent breakthroughs in Natural Language Processing (NLP) have been driv...