Future Sight: Dynamic Story Generation with Large Pretrained Language Models

12/20/2022
by   Brian D. Zimmerman, et al.
0

Recent advances in deep learning research, such as transformers, have bolstered the ability for automated agents to generate creative texts similar to those that a human would write. By default, transformer decoders can only generate new text with respect to previously generated text. The output distribution of candidate tokens at any position is conditioned on previously selected tokens using a self-attention mechanism to emulate the property of autoregression. This is inherently limiting for tasks such as controllable story generation where it may be necessary to condition on future plot events when writing a story. In this work, we propose Future Sight, a method for finetuning a pretrained generative transformer on the task of future conditioning. Transformer decoders are typically pretrained on the task of completing a context, one token at a time, by means of self-attention. Future Sight additionally enables a decoder to attend to an encoded future plot event. This motivates the decoder to expand on the context in a way that logically concludes with the provided future. During inference, the future plot event can be written by a human author to steer the narrative being generated in a certain direction. We evaluate the efficacy of our approach on a story generation task with human evaluators.

READ FULL TEXT
research
05/13/2018

Hierarchical Neural Story Generation

We explore story generation: creative systems that can build coherent an...
research
12/16/2021

Goal-Directed Story Generation: Augmenting Generative Language Models with Reinforcement Learning

The advent of large pre-trained generative language models has provided ...
research
09/09/2019

Counterfactual Story Reasoning and Generation

Counterfactual reasoning requires predicting how alternative events, con...
research
09/13/2022

StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation

Recent advances in text-to-image synthesis have led to large pretrained ...
research
06/07/2022

Plot Writing From Pre-Trained Language Models

Pre-trained language models (PLMs) fail to generate long-form narrative ...
research
07/19/2022

Relational Future Captioning Model for Explaining Likely Collisions in Daily Tasks

Domestic service robots that support daily tasks are a promising solutio...
research
04/05/2020

Semantics of the Unwritten

The semantics of a text is manifested not only by what is read, but also...

Please sign up or login with your details

Forgot password? Click here to reset