Facts2Story: Controlling Text Generation by Key Facts

12/08/2020
by   Eyal Orbach, et al.
0

Recent advancements in self-attention neural network architectures have raised the bar for open-ended text generation. Yet, while current methods are capable of producing a coherent text which is several hundred words long, attaining control over the content that is being generated – as well as evaluating it – are still open questions. We propose a controlled generation task which is based on expanding a sequence of facts, expressed in natural language, into a longer narrative. We introduce human-based evaluation metrics for this task, as well as a method for deriving a large training dataset. We evaluate three methods on this task, based on fine-tuning pre-trained models. We show that while auto-regressive, unidirectional Language Models such as GPT2 produce better fluency, they struggle to adhere to the requested facts. We propose a plan-and-cloze model (using fine-tuned XLNet) which produces competitive fluency while adhering to the requested content.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/23/2022

What Makes Data-to-Text Generation Hard for Pretrained Language Models?

Expressing natural language descriptions of structured facts or relation...
research
12/09/2022

Plug-and-Play Recipe Generation with Content Planning

Recent pre-trained language models have shown promising capabilities in ...
research
11/25/2022

CodeExp: Explanatory Code Document Generation

Developing models that can automatically generate detailed code explanat...
research
02/08/2023

Auto-Learning: An Adversarial Process of Two Pre-trained Models for Natural Language Generation

Pre-trained models have been used in many fields in recent years, rangin...
research
12/10/2021

Discourse-Aware Prompt Design for Text Generation

Current efficient fine-tuning methods (e.g., adapters, prefix-tuning, et...
research
11/22/2022

Linear Interpolation In Parameter Space is Good Enough for Fine-Tuned Language Models

The simplest way to obtain continuous interpolation between two points i...
research
02/12/2020

Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models

Automated evaluation of open domain natural language generation (NLG) mo...

Please sign up or login with your details

Forgot password? Click here to reset