DeepAI AI Chat
Log In Sign Up

Order in the Court: Explainable AI Methods Prone to Disagreement

by   Michael Neely, et al.

In Natural Language Processing, feature-additive explanation methods quantify the independent contribution of each input token towards a model's decision. By computing the rank correlation between attention weights and the scores produced by a small sample of these methods, previous analyses have sought to either invalidate or support the role of attention-based explanations as a faithful and plausible measure of salience. To investigate what measures of rank correlation can reliably conclude, we comprehensively compare feature-additive methods, including attention-based explanations, across several neural architectures and tasks. In most cases, we find that none of our chosen methods agree. Therefore, we argue that rank correlation is largely uninformative and does not measure the quality of feature-additive methods. Additionally, the range of conclusions a practitioner may draw from a single explainability algorithm are limited.


Rethinking Attention-Model Explainability through Faithfulness Violation Test

Attention mechanisms are dominating the explainability of deep models. T...

iBreakDown: Uncertainty of Model Explanations for Non-additive Predictive Models

Explainable Artificial Intelligence (XAI) brings a lot of attention rece...

Multi-Layer Attention-Based Explainability via Transformers for Tabular Data

We propose a graph-oriented attention-based explainability method for ta...

How (Not) To Evaluate Explanation Quality

The importance of explainability is increasingly acknowledged in natural...

Explaining robust additive utility models by sequences of preference swaps

Multicriteria decision analysis aims at supporting a person facing a dec...

SCOUTER: Slot Attention-based Classifier for Explainable Image Recognition

Explainable artificial intelligence is gaining attention. However, most ...