Unified Shapley Framework to Explain Prediction Drift

02/15/2021
by   Aalok Shanbhag, et al.
12

Predictions are the currency of a machine learning model, and to understand the model's behavior over segments of a dataset, or over time, is an important problem in machine learning research and practice. There currently is no systematic framework to understand this drift in prediction distributions over time or between two semantically meaningful slices of data, in terms of the input features and points. We propose GroupShapley and GroupIG (Integrated Gradients), as axiomatically justified methods to tackle this problem. In doing so, we re-frame all current feature/data importance measures based on the Shapley value as essentially problems of distributional comparisons, and unify them under a common umbrella. We axiomatize certain desirable properties of distributional difference, and study the implications of choosing them empirically.

READ FULL TEXT
research
05/12/2020

Handling Concept Drift for Predictions in Business Process Mining

Predictive services nowadays play an important role across all business ...
research
06/16/2020

Consistent Recalibration Models and Deep Calibration

Consistent Recalibration models (CRC) have been introduced to capture in...
research
04/07/2020

Adversarial Validation Approach to Concept Drift Problem in Automated Machine Learning Systems

In automated machine learning systems, concept drift in input data is on...
research
09/07/2021

LEAF: Navigating Concept Drift in Cellular Networks

Operational networks commonly rely on machine learning models for many t...
research
02/10/2020

Explaining Explanations: Axiomatic Feature Interactions for Deep Networks

Recent work has shown great promise in explaining neural network behavio...
research
11/04/2022

Data Models for Dataset Drift Controls in Machine Learning With Images

Camera images are ubiquitous in machine learning research. They also pla...
research
05/28/2023

Reliable and Interpretable Drift Detection in Streams of Short Texts

Data drift is the change in model input data that is one of the key fact...

Please sign up or login with your details

Forgot password? Click here to reset