Entailment as Robust Self-Learner

05/26/2023
by   Jiaxin Ge, et al.
0

Entailment has been recognized as an important metric for evaluating natural language understanding (NLU) models, and recent studies have found that entailment pretraining benefits weakly supervised fine-tuning. In this work, we design a prompting strategy that formulates a number of different NLU tasks as contextual entailment. This approach improves the zero-shot adaptation of pretrained entailment models. Secondly, we notice that self-training entailment-based models with unlabeled data can significantly improve the adaptation performance on downstream tasks. To achieve more stable improvement, we propose the Simple Pseudo-Label Editing (SimPLE) algorithm for better pseudo-labeling quality in self-training. We also found that both pretrained entailment-based models and the self-trained models are robust against adversarial evaluation data. Experiments on binary and multi-class classification tasks show that SimPLE leads to more robust self-training results, indicating that the self-trained entailment models are more efficient and trustworthy than large language models on language understanding tasks.

READ FULL TEXT
research
11/26/2018

Visual Entailment Task for Visually-Grounded Language Learning

We introduce a new inference task - Visual Entailment (VE) - which diffe...
research
03/20/2022

Entailment Relation Aware Paraphrase Generation

We introduce a new task of entailment relation aware paraphrase generati...
research
05/13/2023

SCENE: Self-Labeled Counterfactuals for Extrapolating to Negative Examples

Detecting negatives (such as non-entailment relationships, unanswerable ...
research
03/10/2023

Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoning

Due to their similarity-based learning objectives, pretrained sentence e...
research
09/08/2021

Continuous Entailment Patterns for Lexical Inference in Context

Combining a pretrained language model (PLM) with textual patterns has be...
research
10/28/2022

Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTE

Figurative language (e.g., "he flew like the wind") is challenging to un...
research
05/24/2023

ChatGPT and Simple Linguistic Inferences: Blind Spots and Blinds

This paper sheds light on the limitations of ChatGPT's understanding cap...

Please sign up or login with your details

Forgot password? Click here to reset