DeepAI AI Chat
Log In Sign Up

Randomized Deep Structured Prediction for Discourse-Level Processing

by   Manuel Widmoser, et al.

Expressive text encoders such as RNNs and Transformer Networks have been at the center of NLP models in recent work. Most of the effort has focused on sentence-level tasks, capturing the dependencies between words in a single sentence, or pairs of sentences. However, certain tasks, such as argumentation mining, require accounting for longer texts and complicated structural dependencies between them. Deep structured prediction is a general framework to combine the complementary strengths of expressive neural encoders and structured inference for highly structured domains. Nevertheless, when the need arises to go beyond sentences, most work relies on combining the output scores of independently trained classifiers. One of the main reasons for this is that constrained inference comes at a high computational cost. In this paper, we explore the use of randomized inference to alleviate this concern and show that we can efficiently leverage deep structured prediction and expressive neural encoders for a set of tasks involving complicated argumentative structures.


page 1

page 2

page 3

page 4


Modeling Content and Context with Deep Relational Learning

Building models for realistic natural language tasks requires dealing wi...

Composition of Sentence Embeddings:Lessons from Statistical Relational Learning

Various NLP problems -- such as the prediction of sentence similarity, e...

Evaluation Benchmarks and Learning Criteriafor Discourse-Aware Sentence Representations

Prior work on pretrained sentence embeddings and benchmarks focus on the...

Transferring Semantic Knowledge Into Language Encoders

We introduce semantic form mid-tuning, an approach for transferring sema...

Hierarchical Transformer Networks for Longitudinal Clinical Document Classification

We present the Hierarchical Transformer Networks for modeling long-term ...

Zero-Shot Multi-Label Topic Inference with Sentence Encoders

Sentence encoders have indeed been shown to achieve superior performance...

Tighter Bounds on the Expressivity of Transformer Encoders

Characterizing neural networks in terms of better-understood formal syst...