ALICE: Active Learning with Contrastive Natural Language Explanations

09/22/2020
by   Weixin Liang, et al.
15

Training a supervised neural network classifier typically requires many annotated training samples. Collecting and annotating a large number of data points are costly and sometimes even infeasible. Traditional annotation process uses a low-bandwidth human-machine communication interface: classification labels, each of which only provides several bits of information. We propose Active Learning with Contrastive Explanations (ALICE), an expert-in-the-loop training framework that utilizes contrastive natural language explanations to improve data efficiency in learning. ALICE learns to first use active learning to select the most informative pairs of label classes to elicit contrastive natural language explanations from experts. Then it extracts knowledge from these explanations using a semantic parser. Finally, it incorporates the extracted knowledge through dynamically changing the learning model's structure. We applied ALICE in two visual recognition tasks, bird species classification and social relationship classification. We found by incorporating contrastive explanations, our models outperform baseline models that are trained with 40-100 explanation leads to similar performance gain as adding 13-30 labeled training data points.

READ FULL TEXT

page 1

page 3

page 6

page 9

research
05/27/2018

Semantic Explanations of Predictions

The main objective of explanations is to transmit knowledge to humans. T...
research
09/08/2021

Active Learning by Acquiring Contrastive Examples

Common acquisition functions for active learning use either uncertainty ...
research
05/10/2018

Training Classifiers with Natural Language Explanations

Training accurate classifiers requires many labels, but each label provi...
research
06/22/2011

Acquiring Word-Meaning Mappings for Natural Language Interfaces

This paper focuses on a system, WOLFIE (WOrd Learning From Interpreted E...
research
07/06/2023

Contrast Is All You Need

In this study, we analyze data-scarce classification scenarios, where av...
research
09/16/2021

Let the CAT out of the bag: Contrastive Attributed explanations for Text

Contrastive explanations for understanding the behavior of black box mod...
research
02/08/2023

CRL+: A Novel Semi-Supervised Deep Active Contrastive Representation Learning-Based Text Classification Model for Insurance Data

Financial sector and especially the insurance industry collect vast volu...

Please sign up or login with your details

Forgot password? Click here to reset