DeepAI AI Chat
Log In Sign Up

Remedying BiLSTM-CNN Deficiency in Modeling Cross-Context for NER

by   Peng-Hsuan Li, et al.
Academia Sinica

Recent researches prevalently used BiLSTM-CNN as a core module for NER in a sequence-labeling setup. This paper formally shows the limitation of BiLSTM-CNN encoders in modeling cross-context patterns for each word, i.e., patterns crossing past and future for a specific time step. Two types of cross-structures are used to remedy the problem: A BiLSTM variant with cross-link between layers; a multi-head self-attention mechanism. These cross-structures bring consistent improvements across a wide range of NER domains for a core system using BiLSTM-CNN without additional gazetteers, POS taggers, language-modeling, or multi-task supervision. The model surpasses comparable previous models on OntoNotes 5.0 and WNUT 2017 by 1.4 especially improving emerging, complex, confusing, and multi-token entity mentions, showing the importance of remedying the core module of NER.


Why Attention? Analyzing and Remedying BiLSTM Deficiency in Modeling Cross-Context for NER

State-of-the-art approaches of NER have used sequence-labeling BiLSTM as...

NER-MQMRC: Formulating Named Entity Recognition as Multi Question Machine Reading Comprehension

NER has been traditionally formulated as a sequence labeling task. Howev...

MELM: Data Augmentation with Masked Entity Language Modeling for Cross-lingual NER

Data augmentation for cross-lingual NER requires fine-grained control ov...

One Model for All Domains: Collaborative Domain-Prefix Tuning for Cross-Domain NER

Cross-domain NER is a challenging task to address the low-resource probl...

Neural Modeling for Named Entities and Morphology (NEMO^2)

Named Entity Recognition (NER) is a fundamental NLP task, commonly formu...

TMR: Evaluating NER Recall on Tough Mentions

We propose the Tough Mentions Recall (TMR) metrics to supplement traditi...

Code Repositories


CKIP Neural Chinese Word Segmentation, POS Tagging, and NER

view repo