Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve

by   Oshin Agarwal, et al.

Named Entity Recognition systems achieve remarkable performance on domains such as English news. It is natural to ask: What are these models actually learning to achieve this? Are they merely memorizing the names themselves? Or are they capable of interpreting the text and inferring the correct entity type from the linguistic context? We examine these questions by contrasting the performance of several variants of LSTM-CRF architectures for named entity recognition, with some provided only representations of the context as features. We also perform similar experiments for BERT. We find that context representations do contribute to system performance, but that the main factor driving high performance is learning the name tokens themselves. We enlist human annotators to evaluate the feasibility of inferring entity types from the context alone and find that, while people are not able to infer the entity type either for the majority of the errors made by the context-only system, there is some room for improvement. A system should be able to recognize any name in a predictive context correctly and our experiments indicate that current systems may be further improved by such capability.


page 1

page 2

page 3

page 4


Morphological Embeddings for Named Entity Recognition in Morphologically Rich Languages

In this work, we present new state-of-the-art results of 93.59, for Turk...

Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition

Interpretable rationales for model predictions play a critical role in p...

WCL-BBCD: A Contrastive Learning and Knowledge Graph Approach to Named Entity Recognition

Named Entity Recognition task is one of the core tasks of information ex...

Statement networks: a power structure narrative as depicted by newspapers

We report a data mining pipeline and subsequent analysis to understand t...

Named Person Coreference in English News

People are often entities of interest in tasks such as search and inform...

What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?

We evaluate named entity representations of BERT-based NLP models by inv...

Unsilencing Colonial Archives via Automated Entity Recognition

Colonial archives are at the center of increased interest from a variety...

Please sign up or login with your details

Forgot password? Click here to reset