Gradient Imitation Reinforcement Learning for General Low-Resource Information Extraction

11/11/2022
by   Xuming Hu, et al.
0

Information Extraction (IE) aims to extract structured information from heterogeneous sources. IE from natural language texts include sub-tasks such as Named Entity Recognition (NER), Relation Extraction (RE), and Event Extraction (EE). Most IE systems require comprehensive understandings of sentence structure, implied semantics, and domain knowledge to perform well; thus, IE tasks always need adequate external resources and annotations. However, it takes time and effort to obtain more human annotations. Low-Resource Information Extraction (LRIE) strives to use unsupervised data, reducing the required resources and human annotation. In practice, existing systems either utilize self-training schemes to generate pseudo labels that will cause the gradual drift problem, or leverage consistency regularization methods which inevitably possess confirmation bias. To alleviate confirmation bias due to the lack of feedback loops in existing LRIE learning paradigms, we develop a Gradient Imitation Reinforcement Learning (GIRL) method to encourage pseudo-labeled data to imitate the gradient descent direction on labeled data, which can force pseudo-labeled data to achieve better optimization capabilities similar to labeled data. Based on how well the pseudo-labeled data imitates the instructive gradient descent direction obtained from labeled data, we design a reward to quantify the imitation process and bootstrap the optimization capability of pseudo-labeled data through trial and error. In addition to learning paradigms, GIRL is not limited to specific sub-tasks, and we leverage GIRL to solve all IE sub-tasks (named entity recognition, relation extraction, and event extraction) in low-resource settings (semi-supervised IE and few-shot IE).

READ FULL TEXT

page 4

page 10

page 14

research
09/14/2021

Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction

Low-resource Relation Extraction (LRE) aims to extract relation facts fr...
research
03/28/2022

Using Domain Knowledge for Low Resource Named Entity Recognition

In recent years, named entity recognition has always been a popular rese...
research
03/18/2020

Distant Supervision and Noisy Label Learning for Low Resource Named Entity Recognition: A Study on Hausa and Yorùbá

The lack of labeled training data has limited the development of natural...
research
03/15/2018

A Study of Recent Contributions on Information Extraction

This paper reports on modern approaches in Information Extraction (IE) a...
research
10/19/2022

Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study

This paper presents an empirical study to build relation extraction syst...
research
04/28/2023

RexUIE: A Recursive Method with Explicit Schema Instructor for Universal Information Extraction

Universal Information Extraction (UIE) is an area of interest due to the...
research
08/19/2021

QUEACO: Borrowing Treasures from Weakly-labeled Behavior Data for Query Attribute Value Extraction

We study the problem of query attribute value extraction, which aims to ...

Please sign up or login with your details

Forgot password? Click here to reset