DeepAI AI Chat
Log In Sign Up

The Unreliability of Explanations in Few-Shot In-Context Learning

by   Xi Ye, et al.
The University of Texas at Austin

How can prompting a large language model like GPT-3 with explanations improve in-context learning? We focus specifically on two NLP tasks that involve reasoning over text, namely question answering and natural language inference. Including explanations in the prompt and having the model generate them does not consistently improve performance in the settings we study, contrary to recent results on symbolic reasoning tasks (Nye et al., 2021; Wei et al., 2022). Despite careful prompting, explanations generated by GPT-3 may not even be factually grounded in the input, even on simple tasks with straightforward extractive explanations. However, these flawed explanations can still be useful as a way to verify GPT-3's predictions post-hoc. Through analysis in three settings, we show that explanations judged as good by humans–those that are logically consistent with the input and the prediction–usually indicate more accurate predictions. Following these observations, we present a framework for calibrating model predictions based on the reliability of the explanations. Our framework trains calibrators using automatically extracted scores that approximately assess the reliability of explanations, which helps improve performance across three different datasets.


Investigating the Effect of Natural Language Explanations on Out-of-Distribution Generalization in Few-shot NLI

Although neural models have shown strong performance in datasets such as...

MaNtLE: Model-agnostic Natural Language Explainer

Understanding the internal reasoning behind the predictions of machine l...

Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach

Humans work together to solve common problems by having discussions, exp...

Natural Language Interaction with Explainable AI Models

This paper presents an explainable AI (XAI) system that provides explana...

FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation

Natural language (NL) explanations of model predictions are gaining popu...

Evaluation of Automatically Constructed Word Meaning Explanations

Preparing exact and comprehensive word meaning explanations is one of th...