On the Interaction of Belief Bias and Explanations

06/29/2021
by   Ana Valeria Gonzalez, et al.
1

A myriad of explainability methods have been proposed in recent years, but there is little consensus on how to evaluate them. While automatic metrics allow for quick benchmarking, it isn't clear how such metrics reflect human interaction with explanations. Human evaluation is of paramount importance, but previous protocols fail to account for belief biases affecting human performance, which may lead to misleading conclusions. We provide an overview of belief bias, its role in human evaluation, and ideas for NLP practitioners on how to account for it. For two experimental paradigms, we present a case study of gradient-based explainability introducing simple ways to account for humans' prior beliefs: models of varying quality and adversarial examples. We show that conclusions about the highest performing methods change when introducing such controls, pointing to the importance of accounting for belief bias in evaluation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/14/2020

Measuring and improving the quality of visual explanations

The ability of to explain neural network decisions goes hand in hand wit...
research
03/06/2023

IFAN: An Explainability-Focused Interaction Framework for Humans and NLP Models

Interpretability and human oversight are fundamental pillars of deployin...
research
11/09/2022

On the Robustness of Explanations of Deep Neural Network Models: A Survey

Explainability has been widely stated as a cornerstone of the responsibl...
research
02/23/2023

The Generalizability of Explanations

Due to the absence of ground truth, objective evaluation of explainabili...
research
07/11/2023

Cognitive Bias and Belief Revision

In this paper we formalise three types of cognitive bias within the fram...
research
06/17/2022

Explainability's Gain is Optimality's Loss? – How Explanations Bias Decision-making

Decisions in organizations are about evaluating alternatives and choosin...
research
04/26/2023

Are Explainability Tools Gender Biased? A Case Study on Face Presentation Attack Detection

Face recognition (FR) systems continue to spread in our daily lives with...

Please sign up or login with your details

Forgot password? Click here to reset