Randomized Deep Structured Prediction for Discourse-Level Processing

by   Manuel Widmoser, et al.

Expressive text encoders such as RNNs and Transformer Networks have been at the center of NLP models in recent work. Most of the effort has focused on sentence-level tasks, capturing the dependencies between words in a single sentence, or pairs of sentences. However, certain tasks, such as argumentation mining, require accounting for longer texts and complicated structural dependencies between them. Deep structured prediction is a general framework to combine the complementary strengths of expressive neural encoders and structured inference for highly structured domains. Nevertheless, when the need arises to go beyond sentences, most work relies on combining the output scores of independently trained classifiers. One of the main reasons for this is that constrained inference comes at a high computational cost. In this paper, we explore the use of randomized inference to alleviate this concern and show that we can efficiently leverage deep structured prediction and expressive neural encoders for a set of tasks involving complicated argumentative structures.



page 1

page 2

page 3

page 4


Modeling Content and Context with Deep Relational Learning

Building models for realistic natural language tasks requires dealing wi...

Composition of Sentence Embeddings:Lessons from Statistical Relational Learning

Various NLP problems -- such as the prediction of sentence similarity, e...

Paragraph-based Transformer Pre-training for Multi-Sentence Inference

Inference tasks such as answer sentence selection (AS2) or fact verifica...

Transferring Semantic Knowledge Into Language Encoders

We introduce semantic form mid-tuning, an approach for transferring sema...

Hierarchical Transformer Networks for Longitudinal Clinical Document Classification

We present the Hierarchical Transformer Networks for modeling long-term ...

Learning to Compose Words into Sentences with Reinforcement Learning

We use reinforcement learning to learn tree-structured neural networks f...

Scalable Micro-planned Generation of Discourse from Structured Data

We present a framework for generating natural language description from ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.