DeepAI AI Chat
Log In Sign Up

Larger-Context Tagging: When and Why Does It Work?

by   Jinlan Fu, et al.

The development of neural networks and pretraining techniques has spawned many sentence-level tagging systems that achieved superior performance on typical benchmarks. However, a relatively less discussed topic is what if more context information is introduced into current top-scoring tagging systems. Although several existing works have attempted to shift tagging systems from sentence-level to document-level, there is still no consensus conclusion about when and why it works, which limits the applicability of the larger-context approach in tagging tasks. In this paper, instead of pursuing a state-of-the-art tagging system by architectural exploration, we focus on investigating when and why the larger-context training, as a general strategy, can work. To this end, we conduct a thorough comparative study on four proposed aggregators for context information collecting and present an attribute-aided evaluation method to interpret the improvement brought by larger-context training. Experimentally, we set up a testbed based on four tagging tasks and thirteen datasets. Hopefully, our preliminary observations can deepen the understanding of larger-context training and enlighten more follow-up works on the use of contextual information.


Global Attention for Name Tagging

Many name tagging approaches use local contextual information with much ...

Improving Part-of-Speech Tagging for NLP Pipelines

This paper outlines the results of sentence level linguistics based rule...

LemMED: Fast and Effective Neural Morphological Analysis with Short Context Windows

We present LemMED, a character-level encoder-decoder for contextual morp...

A General-Purpose Tagger with Convolutional Neural Networks

We present a general-purpose tagger based on convolutional neural networ...

Music Auto-tagging Using CNNs and Mel-spectrograms With Reduced Frequency and Time Resolution

Automatic tagging of music is an important research topic in Music Infor...

Supervised Neural Models Revitalize the Open Relation Extraction

Open relation extraction (ORE) remains a challenge to obtain a semantic ...

Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging

In this paper we show that reporting a single performance score is insuf...