Shapley-based explainability on the data manifold

by   Christopher Frye, et al.

Explainability in machine learning is crucial for iterative model development, compliance with regulation, and providing operational nuance to model predictions. Shapley values provide a general framework for explainability by attributing a model's output prediction to its input features in a mathematically principled and model-agnostic way. However, practical implementations of the Shapley framework make an untenable assumption: that the model's input features are uncorrelated. In this work, we articulate the dangers of this assumption and introduce two solutions for computing Shapley explanations that respect the data manifold. One solution, based on generative modelling, provides flexible access to on-manifold data imputations, while the other directly learns the Shapley value function in a supervised way, providing performance and stability at the cost of flexibility. While the commonly used “off-manifold” Shapley values can (i) break symmetries in the data, (ii) give rise to misleading wrong-sign explanations, and (iii) lead to uninterpretable explanations in high-dimensional data, our approach to on-manifold explainability demonstrably overcomes each of these problems.


Manifold Restricted Interventional Shapley Values

Shapley values are model-agnostic methods for explaining model predictio...

Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability

Explaining AI systems is fundamental both to the development of high per...

Human-interpretable model explainability on high-dimensional data

The importance of explainability in machine learning continues to grow, ...

Threading the Needle of On and Off-Manifold Value Functions for Shapley Explanations

A popular explainable AI (XAI) approach to quantify feature importance o...

A Refutation of Shapley Values for Explainability

Recent work demonstrated the existence of Boolean functions for which Sh...

L'explicabilité au service de l'extraction de connaissances : application à des données médicales

The use of machine learning has increased dramatically in the last decad...

NeuroExplainer: Fine-Grained Attention Decoding to Uncover Cortical Development Patterns of Preterm Infants

Deploying reliable deep learning techniques in interdisciplinary applica...

Please sign up or login with your details

Forgot password? Click here to reset