DeepAI AI Chat
Log In Sign Up

End-to-end Named Entity Recognition and Relation Extraction using Pre-trained Language Models

by   John Giorgi, et al.

Named entity recognition (NER) and relation extraction (RE) are two important tasks in information extraction and retrieval (IE & IR). Recent work has demonstrated that it is beneficial to learn these tasks jointly, which avoids the propagation of error inherent in pipeline-based systems and improves performance. However, state-of-the-art joint models typically rely on external natural language processing (NLP) tools, such as dependency parsers, limiting their usefulness to domains (e.g. news) where those tools perform well. The few neural, end-to-end models that have been proposed are trained almost completely from scratch. In this paper, we propose a neural, end-to-end model for jointly extracting entities and their relations which does not rely on external NLP tools and which integrates a large, pre-trained language model. Because the bulk of our model's parameters are pre-trained and we eschew recurrence for self-attention, our model is fast to train. On 5 datasets across 3 domains, our model matches or exceeds state-of-the-art performance, sometimes by a large margin.


page 1

page 2

page 3

page 4


VTCC-NLP at NL4Opt competition subtask 1: An Ensemble Pre-trained language models for Named Entity Recognition

We propose a combined three pre-trained language models (XLM-R, BART, an...

Joint entity recognition and relation extraction as a multi-head selection problem

State-of-the-art models for joint entity recognition and relation extrac...

End-to-End Models for Chemical-Protein Interaction Extraction: Better Tokenization and Span-Based Pipeline Strategies

End-to-end relation extraction (E2ERE) is an important task in informati...

Contextualization and Generalization in Entity and Relation Extraction

During the past decade, neural networks have become prominent in Natural...

Autoregressive Structured Prediction with Language Models

Recent years have seen a paradigm shift in NLP towards using pretrained ...

Neural Metric Learning for Fast End-to-End Relation Extraction

Relation extraction (RE) is an indispensable information extraction task...