Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction

04/19/2019
by   Maosen Zhang, et al.
0

In recent years there is surge of interest in applying distant supervision (DS) to automatically generate training data for relation extraction. However, despite extensive efforts have been done on constructing advanced neural models, our experiments reveal that these neural models demonstrate only similar (or even worse) performance as compared with simple, feature-based methods. In this paper, we conduct thorough analysis to answer the question what other factors limit the performance of DS-trained neural models? Our results show that shifted labeled distribution commonly exists on real-world DS datasets, and impact of such issue is further validated using synthetic datasets for all models. Building upon the new insight, we develop a simple yet effective adaptation method for DS methods, called bias adjustment, to update models learned over source domain (i.e., DS training set) with label distribution statistics estimated on target domain (i.e., evaluation set). Experiments demonstrate that bias adjustment achieves consistent performance gains on all methods, especially on neural models, with up to a 22 improvement.

READ FULL TEXT
research
06/06/2023

FinRED: A Dataset for Relation Extraction in Financial Domain

Relation extraction models trained on a source domain cannot be applied ...
research
04/14/2022

FREDA: Flexible Relation Extraction Data Annotation

To effectively train accurate Relation Extraction models, sufficient and...
research
03/29/2019

Structured Minimally Supervised Learning for Neural Relation Extraction

We present an approach to minimally supervised relation extraction that ...
research
10/30/2020

Towards Accurate and Consistent Evaluation: A Dataset for Distantly-Supervised Relation Extraction

In recent years, distantly-supervised relation extraction has achieved a...
research
05/28/2020

Relation Extraction with Explanation

Recent neural models for relation extraction with distant supervision al...
research
04/10/2019

Simple BERT Models for Relation Extraction and Semantic Role Labeling

We present simple BERT-based models for relation extraction and semantic...
research
04/30/2020

TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task

TACRED (Zhang et al., 2017) is one of the largest, most widely used crow...

Please sign up or login with your details

Forgot password? Click here to reset