Improving Distantly-Supervised Relation Extraction through BERT-based Label Instance Embeddings

02/01/2021
by   Despina Christou, et al.
0

Distantly-supervised relation extraction (RE) is an effective method to scale RE to large corpora but suffers from noisy labels. Existing approaches try to alleviate noise through multi-instance learning and by providing additional information, but manage to recognize mainly the top frequent relations, neglecting those in the long-tail. We propose REDSandT (Relation Extraction with Distant Supervision and Transformers), a novel distantly-supervised transformer-based RE method, that manages to capture a wider set of relations through highly informative instance and label embeddings for RE, by exploiting BERT's pre-trained model, and the relationship between labels and entities, respectively. We guide REDSandT to focus solely on relational tokens by fine-tuning BERT on a structured input, including the sub-tree connecting an entity pair and the entities' types. Using the extracted informative vectors, we shape label embeddings, which we also use as attention mechanism over instances to further reduce noise. Finally, we represent sentences by concatenating relation and instance embeddings. Experiments in the NYT-10 dataset show that REDSandT captures a broader set of relations with higher confidence, achieving state-of-the-art AUC (0.424).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/19/2019

Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction

Distantly supervised relation extraction is widely used to extract relat...
research
10/14/2021

A Simple, Strong and Robust Baseline for Distantly Supervised Relation Extraction

Distantly supervised relation extraction (DS-RE) is generally framed as ...
research
04/15/2021

A Sample-Based Training Method for Distantly Supervised Relation Extraction with Pre-Trained Transformers

Multiple instance learning (MIL) has become the standard learning paradi...
research
12/22/2016

Noise Mitigation for Neural Entity Typing and Relation Extraction

In this paper, we address two different types of noise in information ex...
research
11/01/2019

Deep Bidirectional Transformers for Relation Extraction without Supervision

We present a novel framework to deal with relation extraction tasks in c...
research
09/08/2020

High-throughput relation extraction algorithm development associating knowledge articles and electronic health records

Objective: Medical relations are the core components of medical knowledg...
research
02/04/2019

Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers

Most approaches to extraction multiple relations from a paragraph requir...

Please sign up or login with your details

Forgot password? Click here to reset