A Latent Variable Recurrent Neural Network for Discourse Relation Language Models

03/07/2016
by   Yangfeng Ji, et al.
0

This paper presents a novel latent variable recurrent neural network architecture for jointly modeling sequences of words and (possibly latent) discourse relations between adjacent sentences. A recurrent neural network generates individual words, thus reaping the benefits of discriminatively-trained vector representations. The discourse relations are represented with a latent variable, which can be predicted or marginalized, depending on the task. The resulting model can therefore employ a training objective that includes not only discourse relation classification, but also word prediction. As a result, it outperforms state-of-the-art alternatives for two tasks: implicit discourse relation classification in the Penn Discourse Treebank, and dialog act classification in the Switchboard corpus. Furthermore, by marginalizing over latent discourse relations at test time, we obtain a discourse informed language model, which improves over a strong LSTM baseline.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/12/2016

Variational Neural Discourse Relation Recognizer

Implicit discourse relation recognition is a crucial component for autom...
research
08/30/2019

Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs

Generating a long, coherent text such as a paragraph requires a high-lev...
research
04/26/2017

A Recurrent Neural Model with Attention for the Recognition of Chinese Implicit Discourse Relations

We introduce an attention-based Bi-LSTM for Chinese implicit discourse r...
research
11/05/2016

Reference-Aware Language Models

We propose a general class of language models that treat reference as an...
research
06/14/2016

Shallow Discourse Parsing Using Distributed Argument Representations and Bayesian Optimization

This paper describes the Georgia Tech team's approach to the CoNLL-2016 ...
research
02/13/2015

A Linear Dynamical System Model for Text

Low dimensional representations of words allow accurate NLP models to be...
research
01/08/2020

A Neural Approach to Discourse Relation Signal Detection

Previous data-driven work investigating the types and distributions of d...

Please sign up or login with your details

Forgot password? Click here to reset