Computationally Efficient NER Taggers with Combined Embeddings and Constrained Decoding

01/05/2020
by   Brian Lester, et al.
0

Current State-of-the-Art models in Named Entity Recognition (NER) are neural models with a Conditional Random Field (CRF) as the final network layer, and pre-trained "contextual embeddings". The CRF layer is used to facilitate global coherence between labels, and the contextual embeddings provide a better representation of words in context. However, both of these improvements come at a high computational cost. In this work, we explore two simple techniques that substantially improve NER performance over a strong baseline with negligible cost. First, we use multiple pre-trained embeddings as word representations via concatenation. Second, we constrain the tagger, trained using a cross-entropy loss, during decoding to eliminate illegal transitions. While training a tagger on CoNLL 2003 we find a 786% speed-up over a contextual embeddings-based tagger without sacrificing strong performance. We also show that the concatenation technique works across multiple tasks and datasets. We analyze aspects of similarity and coverage between pre-trained embeddings and the dynamics of tag co-occurrence to explain why these techniques work. We provide an open source implementation of our tagger using these techniques in three popular deep learning frameworks — TensorFlow, Pytorch, and DyNet.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/09/2020

Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers

Current state-of-the-art models for named entity recognition (NER) are n...
research
09/30/2020

Multiple Word Embeddings for Increased Diversity of Representation

Most state-of-the-art models in natural language processing (NLP) are ne...
research
07/05/2019

Improving Chemical Named Entity Recognition in Patents with Contextualized Word Embeddings

Chemical patents are an important resource for chemical information. How...
research
09/14/2017

Synapse at CAp 2017 NER challenge: Fasttext CRF

We present our system for the CAp 2017 NER challenge which is about name...
research
04/10/2020

One Model to Recognize Them All: Marginal Distillation from NER Models with Different Tag Sets

Named entity recognition (NER) is a fundamental component in the modern ...
research
03/10/2020

Adaptive Name Entity Recognition under Highly Unbalanced Data

For several purposes in Natural Language Processing (NLP), such as Infor...
research
05/04/2022

RecipeSnap – a lightweight image-to-recipe model

In this paper we want to address the problem of automation for recogniti...

Please sign up or login with your details

Forgot password? Click here to reset