Latent SHAP: Toward Practical Human-Interpretable Explanations

by   Ron Bitton, et al.

Model agnostic feature attribution algorithms (such as SHAP and LIME) are ubiquitous techniques for explaining the decisions of complex classification models, such as deep neural networks. However, since complex classification models produce superior performance when trained on low-level (or encoded) features, in many cases, the explanations generated by these algorithms are neither interpretable nor usable by humans. Methods proposed in recent studies that support the generation of human-interpretable explanations are impractical, because they require a fully invertible transformation function that maps the model's input features to the human-interpretable features. In this work, we introduce Latent SHAP, a black-box feature attribution framework that provides human-interpretable explanations, without the requirement for a fully invertible transformation function. We demonstrate Latent SHAP's effectiveness using (1) a controlled experiment where invertible transformation functions are available, which enables robust quantitative evaluation of our method, and (2) celebrity attractiveness classification (using the CelebA dataset) where invertible transformation functions are not available, which enables thorough qualitative evaluation of our method.


page 2

page 5

page 6

page 9


What's in the box? Explaining the black-box model through an evaluation of its interpretable features

Algorithms are powerful and necessary tools behind a large part of the i...

Interpretable Neural Networks with Frank-Wolfe: Sparse Relevance Maps and Relevance Orderings

We study the effects of constrained optimization formulations and Frank-...

Making Neural Networks Interpretable with Attribution: Application to Implicit Signals Prediction

Explaining recommendations enables users to understand whether recommend...

Explanations of model predictions with live and breakDown packages

Complex models are commonly used in predictive modeling. In this paper w...

IS-CAM: Integrated Score-CAM for axiomatic-based explanations

Convolutional Neural Networks have been known as black-box models as hum...

Global Explanations of Neural Networks: Mapping the Landscape of Predictions

A barrier to the wider adoption of neural networks is their lack of inte...

Human-interpretable model explainability on high-dimensional data

The importance of explainability in machine learning continues to grow, ...

Please sign up or login with your details

Forgot password? Click here to reset