DeepAI AI Chat
Log In Sign Up

On Shapley Credit Allocation for Interpretability

by   Debraj Basu, et al.

We emphasize the importance of asking the right question when interpreting the decisions of a learning model. We discuss a natural extension of the theoretical machinery from Janzing et. al. 2020, which answers the question "Why did my model predict a person has cancer?" for answering a more involved question, "What caused my model to predict a person has cancer?" While the former quantifies the direct effects of variables on the model, the latter also accounts for indirect effects, thereby providing meaningful insights wherever human beings can reason in terms of cause and effect. We propose three broad categories for interpretations: observational, model-specific and causal each of which are significant in their own right. Furthermore, this paper quantifies feature relevance by weaving different natures of interpretations together with different measures as characteristic functions for Shapley symmetrization. Besides the widely used expected value of the model, we also discuss measures of statistical uncertainty and dispersion as informative candidates, and their merits in generating explanations for each data point, some of which are used in this context for the first time. These measures are not only useful for studying the influence of variables on the model output, but also on the predictive performance of the model, and for that we propose relevant characteristic functions that are also used for the first time.


page 1

page 2

page 3

page 4


Functional Mediation Analysis with an Application to Functional Magnetic Resonance Imaging Data

Causal mediation analysis is widely utilized to separate the causal effe...

Direct and Indirect Effects – An Information Theoretic Perspective

Information theoretic (IT) approaches to quantifying causal influences h...

CXPlain: Causal Explanations for Model Interpretation under Uncertainty

Feature importance estimates that inform users about the degree to which...

Model Explanations under Calibration

Explaining and interpreting the decisions of recommender systems are bec...

Decomposition of Global Feature Importance into Direct and Associative Components (DEDACT)

Global model-agnostic feature importance measures either quantify whethe...