Explainable Artificial Intelligence: How Subsets of the Training Data Affect a Prediction

12/07/2020
by   Andreas Brandsæter, et al.
0

There is an increasing interest in and demand for interpretations and explanations of machine learning models and predictions in various application areas. In this paper, we consider data-driven models which are already developed, implemented and trained. Our goal is to interpret the models and explain and understand their predictions. Since the predictions made by data-driven models rely heavily on the data used for training, we believe explanations should convey information about how the training data affects the predictions. To do this, we propose a novel methodology which we call Shapley values for training data subset importance. The Shapley value concept originates from coalitional game theory, developed to fairly distribute the payout among a set of cooperating players. We extend this to subset importance, where a prediction is explained by treating the subsets of the training data as players in a game where the predictions are the payouts. We describe and illustrate how the proposed method can be useful and demonstrate its capabilities on several examples. We show how the proposed explanations can be used to reveal biasedness in models and erroneous training data. Furthermore, we demonstrate that when predictions are accurately explained in a known situation, then explanations of predictions by simple models correspond to the intuitive explanations. We argue that the explanations enable us to perceive more of the inner workings of the algorithms, and illustrate how models producing similar predictions can be based on very different parts of the training data. Finally, we show how we can use Shapley values for subset importance to enhance our training data acquisition, and by this reducing prediction error.

READ FULL TEXT

page 9

page 11

research
01/13/2020

Consumer-Driven Explanations for Machine Learning Decisions: An Empirical Study of Robustness

Many proposed methods for explaining machine learning predictions are in...
research
07/19/2023

TbExplain: A Text-based Explanation Method for Scene Classification Models with the Statistical Prediction Correction

The field of Explainable Artificial Intelligence (XAI) aims to improve t...
research
12/07/2021

Training Deep Models to be Explained with Fewer Examples

Although deep models achieve high predictive performance, it is difficul...
research
06/27/2023

Delivering Inflated Explanations

In the quest for Explainable Artificial Intelligence (XAI) one of the qu...
research
09/29/2018

Training Machine Learning Models by Regularizing their Explanations

Neural networks are among the most accurate supervised learning methods ...
research
06/20/2022

Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experiments

We develop a new, principled algorithm for estimating the contribution o...
research
09/21/2023

Quantifying Feature Importance of Games and Strategies via Shapley Values

Recent advances in game informatics have enabled us to find strong strat...

Please sign up or login with your details

Forgot password? Click here to reset