DeepAI AI Chat
Log In Sign Up

Generating Narrative Text in a Switching Dynamical System

by   Noah Weber, et al.
Stony Brook University
Johns Hopkins University
United States Naval Academy

Early work on narrative modeling used explicit plans and goals to generate stories, but the language generation itself was restricted and inflexible. Modern methods use language models for more robust generation, but often lack an explicit representation of the scaffolding and dynamics that guide a coherent narrative. This paper introduces a new model that integrates explicit narrative structure with neural language models, formalizing narrative modeling as a Switching Linear Dynamical System (SLDS). A SLDS is a dynamical system in which the latent dynamics of the system (i.e. how the state vector transforms over time) is controlled by top-level discrete switching variables. The switching variables represent narrative structure (e.g., sentiment or discourse states), while the latent state vector encodes information on the current state of the narrative. This probabilistic formulation allows us to control generation, and can be learned in a semi-supervised fashion using both labeled and unlabeled data. Additionally, we derive a Gibbs sampler for our model that can fill in arbitrary parts of the narrative, guided by the switching variables. Our filled-in (English language) narratives outperform several baselines on both automatic and human evaluations.


page 1

page 2

page 3

page 4


Study of Semi-supervised Approaches to Improving English-Mandarin Code-Switching Speech Recognition

In this paper, we present our overall efforts to improve the performance...

Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs

Generating a long, coherent text such as a paragraph requires a high-lev...

Language modeling via stochastic processes

Modern language models can generate high-quality short texts. However, t...

Dynamical Deep Generative Latent Modeling of 3D Skeletal Motion

In this paper, we propose a Bayesian switching dynamical model for segme...

A Deep Generative Model for Code-Switched Text

Code-switching, the interleaving of two or more languages within a sente...

Reference-Aware Language Models

We propose a general class of language models that treat reference as an...