DeepAI AI Chat
Log In Sign Up

Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability

by   Christopher Frye, et al.

Explaining AI systems is fundamental both to the development of high performing models and to the trust placed in them by their users. A general framework for explaining any AI model is provided by the Shapley values that attribute the prediction output to the various model inputs ("features") in a principled and model-agnostic way. The outstanding strength of Shapley values is their combined generality and rigorous foundation: they can be used to explain any AI system, and one always understands their values as the unique attribution method satisfying a set of mathematical axioms. However, as a framework, Shapley values are too restrictive in one significant regard: they ignore all causal structure in the data. We introduce a less-restrictive framework for model-agnostic explainability: "Asymmetric" Shapley values. Asymmetric Shapley values (ASVs) are rigorously founded on a set of axioms, applicable to any AI system, and can flexibly incorporate any causal knowledge known a-priori to be respected by the data. We show through explicit, realistic examples that the ASV framework can be used to (i) improve model explanations by incorporating causal information, (ii) provide an unambiguous test for unfair discrimination based on simple policy articulations, (iii) enable sequentially incremental explanations in time-series models, and (iv) support feature-selection studies without the need for model retraining.


page 1

page 2

page 3

page 4


Shapley-based explainability on the data manifold

Explainability in machine learning is crucial for iterative model develo...

Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models

Shapley values underlie one of the most popular model-agnostic methods w...

Explainability Via Causal Self-Talk

Explaining the behavior of AI systems is an important problem that, in p...

A general framework for scientifically inspired explanations in AI

Explainability in AI is gaining attention in the computer science commun...

WindowSHAP: An Efficient Framework for Explaining Time-series Classifiers based on Shapley Values

Unpacking and comprehending how deep learning algorithms make decisions ...

"How Does It Detect A Malicious App?" Explaining the Predictions of AI-based Android Malware Detector

AI methods have been proven to yield impressive performance on Android m...

Tell me why! – Explanations support learning of relational and causal structure

Explanations play a considerable role in human learning, especially in a...