Theoretical Behavior of XAI Methods in the Presence of Suppressor Variables

06/02/2023
by   Rick Wilming, et al.
0

In recent years, the community of 'explainable artificial intelligence' (XAI) has created a vast body of methods to bridge a perceived gap between model 'complexity' and 'interpretability'. However, a concrete problem to be solved by XAI methods has not yet been formally stated. As a result, XAI methods are lacking theoretical and empirical evidence for the 'correctness' of their explanations, limiting their potential use for quality-control and transparency purposes. At the same time, Haufe et al. (2014) showed, using simple toy examples, that even standard interpretations of linear models can be highly misleading. Specifically, high importance may be attributed to so-called suppressor variables lacking any statistical relation to the prediction target. This behavior has been confirmed empirically for a large array of XAI methods in Wilming et al. (2022). Here, we go one step further by deriving analytical expressions for the behavior of a variety of popular XAI methods on a simple two-dimensional binary classification problem involving Gaussian class-conditional distributions. We show that the majority of the studied approaches will attribute non-zero importance to a non-class-related suppressor feature in the presence of correlated noise. This poses important limitations on the interpretations and conclusions that the outputs of these XAI methods can afford.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/28/2020

The importance of transparency and reproducibility in artificial intelligence research

In their study, McKinney et al. showed the high potential of artificial ...
research
09/19/2019

Highlighting Bias with Explainable Neural-Symbolic Visual Reasoning

Many high-performance models suffer from a lack of interpretability. The...
research
06/22/2023

XAI-TRIS: Non-linear benchmarks to quantify ML explanation performance

The field of 'explainable' artificial intelligence (XAI) has produced hi...
research
04/02/2021

TAPAS at SemEval-2021 Task 9: Reasoning over tables with intermediate pre-training

We present the TAPAS contribution to the Shared Task on Statement Verifi...
research
01/20/2014

A Scalable Conditional Independence Test for Nonlinear, Non-Gaussian Data

Many relations of scientific interest are nonlinear, and even in linear ...
research
06/09/2021

Avoiding Traps in Nonconvex Problems

Iterative projection methods may become trapped at non-solutions when th...

Please sign up or login with your details

Forgot password? Click here to reset