Hybrid Attention-Based Transformer Block Model for Distant Supervision Relation Extraction

by   Yan Xiao, et al.

With an exponential explosive growth of various digital text information, it is challenging to efficiently obtain specific knowledge from massive unstructured text information. As one basic task for natural language processing (NLP), relation extraction aims to extract the semantic relation between entity pairs based on the given text. To avoid manual labeling of datasets, distant supervision relation extraction (DSRE) has been widely used, aiming to utilize knowledge base to automatically annotate datasets. Unfortunately, this method heavily suffers from wrong labelling due to the underlying strong assumptions. To address this issue, we propose a new framework using hybrid attention-based Transformer block with multi-instance learning to perform the DSRE task. More specifically, the Transformer block is firstly used as the sentence encoder to capture syntactic information of sentences, which mainly utilizes multi-head self-attention to extract features from word level. Then, a more concise sentence-level attention mechanism is adopted to constitute the bag representation, aiming to incorporate valid information of each sentence to effectively represent the bag. Experimental results on the public dataset New York Times (NYT) demonstrate that the proposed approach can outperform the state-of-the-art algorithms on the evaluation dataset, which verifies the effectiveness of our model for the DSRE task.


page 1

page 2

page 3

page 4


Self-Attention Enhanced Selective Gate with Entity-Aware Embedding for Distantly Supervised Relation Extraction

Distantly supervised relation extraction intrinsically suffers from nois...

Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions

This paper presents a neural relation extraction method to deal with the...

Distantly-Supervised Neural Relation Extraction with Side Information using BERT

Relation extraction (RE) consists in categorizing the relationship betwe...

BERE: An accurate distantly supervised biomedical entity relation extraction network

Automated entity relation extraction (RE) from literature provides an im...

Semi-Automating Knowledge Base Construction for Cancer Genetics

In this work, we consider the exponentially growing subarea of genetics ...

HiCLRE: A Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction

Distant supervision assumes that any sentence containing the same entity...

Multi-Level Structured Self-Attentions for Distantly Supervised Relation Extraction

Attention mechanisms are often used in deep neural networks for distantl...