Is Shapley Explanation for a model unique?

by   Harsh Kumar, et al.

Shapley value has recently become a popular way to explain the predictions of complex and simple machine learning models. This paper is discusses the factors that influence Shapley value. In particular, we explore the relationship between the distribution of a feature and its Shapley value. We extend our analysis by discussing the difference that arises in Shapley explanation for different predicted outcomes from the same model. Our assessment is that Shapley value for particular feature not only depends on its expected mean but on other moments as well such as variance and there are disagreements for baseline prediction, disagreements for signs and most important feature for different outcomes such as probability, log odds, and binary decision generated using same linear probability model (logit/probit). These disagreements not only stay for local explainability but also affect the global feature importance. We conclude that there is no unique Shapley explanation for a given model. It varies with model outcome (Probability/Log-odds/binary decision such as accept vs reject) and hence model application.


page 6

page 8


Feature Importance versus Feature Influence and What It Signifies for Explainable AI

When used in the context of decision theory, feature importance expresse...

Q-FIT: The Quantifiable Feature Importance Technique for Explainable Machine Learning

We introduce a novel framework to quantify the importance of each input ...

A Baseline for Shapely Values in MLPs: from Missingness to Neutrality

Being able to explain a prediction as well as having a model that perfor...

Delivering Inflated Explanations

In the quest for Explainable Artificial Intelligence (XAI) one of the qu...

On Locality of Local Explanation Models

Shapley values provide model agnostic feature attributions for model out...

Please sign up or login with your details

Forgot password? Click here to reset