Latent Template Induction with Gumbel-CRFs

11/29/2020
by   Yao Fu, et al.
0

Learning to control the structure of sentences is a challenging problem in text generation. Existing work either relies on simple deterministic approaches or RL-based hard structures. We explore the use of structured variational autoencoders to infer latent templates for sentence generation using a soft, continuous relaxation in order to utilize reparameterization for training. Specifically, we propose a Gumbel-CRF, a continuous relaxation of the CRF sampling algorithm using a relaxed Forward-Filtering Backward-Sampling (FFBS) approach. As a reparameterized gradient estimator, the Gumbel-CRF gives more stable gradients than score-function based estimators. As a structured inference network, we show that it learns interpretable templates during training, which allows us to control the decoder during testing. We demonstrate the effectiveness of our methods with experiments on data-to-text generation and unsupervised paraphrase generation.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/30/2018

Learning Neural Templates for Text Generation

While neural, encoder-decoder models have had significant empirical succ...
04/15/2019

Latent Code and Text-based Generative Adversarial Networks for Soft-text Generation

Text generation with generative adversarial networks (GANs) can be divid...
05/29/2019

Latent Space Secrets of Denoising Text-Autoencoders

While neural language models have recently demonstrated impressive perfo...
02/04/2020

Variational Template Machine for Data-to-Text Generation

How to generate descriptions from structured data organized in tables? E...
10/19/2019

Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation

Neural conditional text generation systems have achieved significant pro...
09/09/2020

Discovering Textual Structures: Generative Grammar Induction using Template Trees

Natural language generation provides designers with methods for automati...
11/07/2020

Template Controllable keywords-to-text Generation

This paper proposes a novel neural model for the understudied task of ge...