Learning Task-specific Representation for Novel Words in Sequence Labeling

05/29/2019
by   Minlong Peng, et al.
0

Word representation is a key component in neural-network-based sequence labeling systems. However, representations of unseen or rare words trained on the end task are usually poor for appreciable performance. This is commonly referred to as the out-of-vocabulary (OOV) problem. In this work, we address the OOV problem in sequence labeling using only training data of the task. To this end, we propose a novel method to predict representations for OOV words from their surface-forms (e.g., character sequence) and contexts. The method is specifically designed to avoid the error propagation problem suffered by existing approaches in the same paradigm. To evaluate its effectiveness, we performed extensive empirical studies on four part-of-speech tagging (POS) tasks and four named entity recognition (NER) tasks. Experimental results show that the proposed method can achieve better or competitive performance on the OOV problem compared with existing state-of-the-art methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/04/2016

End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF

State-of-the-art sequence labeling systems traditionally require large a...
research
07/07/2022

Win-Win Cooperation: Bundling Sequence and Span Models for Named Entity Recognition

For Named Entity Recognition (NER), sequence labeling-based and span-bas...
research
11/13/2020

A Survey on Recent Advances in Sequence Labeling from Deep Learning Models

Sequence labeling (SL) is a fundamental research problem encompassing a ...
research
10/29/2018

Learning Better Internal Structure of Words for Sequence Labeling

Character-based neural models have recently proven very useful for many ...
research
04/24/2017

Semi-supervised Multitask Learning for Sequence Labeling

We propose a sequence labeling framework with a secondary training objec...
research
05/14/2020

NAT: Noise-Aware Training for Robust Neural Sequence Labeling

Sequence labeling systems should perform reliably not only under ideal c...
research
05/31/2023

A Global Context Mechanism for Sequence Labeling

Sequential labeling tasks necessitate the computation of sentence repres...

Please sign up or login with your details

Forgot password? Click here to reset