DeepAI AI Chat
Log In Sign Up

Continual Contrastive Finetuning Improves Low-Resource Relation Extraction

by   Wenxuan Zhou, et al.
University of Southern California

Relation extraction (RE), which has relied on structurally annotated corpora for model training, has been particularly challenging in low-resource scenarios and domains. Recent literature has tackled low-resource RE by self-supervised learning, where the solution involves pretraining the relation embedding by RE-based objective and finetuning on labeled data by classification-based objective. However, a critical challenge to this approach is the gap in objectives, which prevents the RE model from fully utilizing the knowledge in pretrained representations. In this paper, we aim at bridging the gap and propose to pretrain and finetune the RE model using consistent objectives of contrastive learning. Since in this kind of representation learning paradigm, one relation may easily form multiple clusters in the representation space, we further propose a multi-center contrastive loss that allows one relation to form multiple clusters to better align with pretraining. Experiments on two document-level RE datasets, BioRED and Re-DocRED, demonstrate the effectiveness of our method. Particularly, when using 1 outperforms PLM-based RE classifier by 10.5 respectively.


page 1

page 2

page 3

page 4


MapRE: An Effective Semantic Mapping Approach for Low-resource Relation Extraction

Neural relation extraction models have shown promising results in recent...

Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction

Low-resource Relation Extraction (LRE) aims to extract relation facts fr...

Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study

This paper presents an empirical study to build relation extraction syst...

Relation Adversarial Network for Low Resource KnowledgeGraph Completion

Knowledge Graph Completion (KGC) has been proposed to improve Knowledge ...

STAD: Self-Training with Ambiguous Data for Low-Resource Relation Extraction

We present a simple yet effective self-training approach, named as STAD,...

Summarization as Indirect Supervision for Relation Extraction

Relation extraction (RE) models have been challenged by their reliance o...

Supervised Graph Contrastive Pretraining for Text Classification

Contrastive pretraining techniques for text classification has been larg...