Generalisation in Named Entity Recognition: A Quantitative Analysis

01/11/2017
by   Isabelle Augenstein, et al.
0

Named Entity Recognition (NER) is a key NLP task, which is all the more challenging on Web and user-generated content with their diverse and continuously changing language. This paper aims to quantify how this diversity impacts state-of-the-art NER methods, by measuring named entity (NE) and context variability, feature sparsity, and their effects on precision and recall. In particular, our findings indicate that NER approaches struggle to generalise in diverse genres with limited training data. Unseen NEs, in particular, play an important role, which have a higher incidence in diverse genres such as social media than in more regular genres such as newswire. Coupled with a higher incidence of unseen features more generally and the lack of large training corpora, this leads to significantly lower F1 scores for diverse genres as compared to more regular ones. We also find that leading systems rely heavily on surface forms found in training data, having problems generalising beyond these, and offer explanations for this observation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/19/2020

A multimodal deep learning approach for named entity recognition from social media

Named Entity Recognition (NER) from social media posts is a challenging ...
research
06/10/2019

Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Task

In the third shared task of the Computational Approaches to Linguistic C...
research
01/25/2021

Recent Trends in Named Entity Recognition (NER)

The availability of large amounts of computer-readable textual data and ...
research
05/10/2018

SlugNERDS: A Named Entity Recognition Tool for Open Domain Dialogue Systems

In dialogue systems, the tasks of named entity recognition (NER) and nam...
research
03/23/2021

TMR: Evaluating NER Recall on Tough Mentions

We propose the Tough Mentions Recall (TMR) metrics to supplement traditi...
research
02/09/2023

Data Augmentation for Robust Character Detection in Fantasy Novels

Named Entity Recognition (NER) is a low-level task often used as a found...
research
07/24/2021

Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition

In this work, we examine the ability of NER models to use contextual inf...

Please sign up or login with your details

Forgot password? Click here to reset