Evaluating Feature Importance Estimates

06/28/2018
by   Sara Hooker, et al.
4

Estimating the influence of a given feature to a model prediction is challenging. We introduce ROAR, RemOve And Retrain, a benchmark to evaluate the accuracy of interpretability methods that estimate input feature importance in deep neural networks. We remove a fraction of input features deemed to be most important according to each estimator and measure the change to the model accuracy upon retraining. The most accurate estimator will identify inputs as important whose removal causes the most damage to model performance relative to all other estimators. This evaluation produces thought-provoking results -- we find that several estimators are less accurate than a random assignment of feature importance. However, averaging a set of squared noisy estimators (a variant of a technique proposed by Smilkov et al. (2017)), leads to significant gains in accuracy for each method considered and far outperforms such a random guess.

READ FULL TEXT

page 3

page 5

page 7

page 8

page 11

page 12

research
03/06/2022

Evaluation of Interpretability Methods and Perturbation Artifacts in Deep Neural Networks

The challenge of interpreting predictions from deep neural networks has ...
research
03/02/2023

Feature Perturbation Augmentation for Reliable Evaluation of Importance Estimators

Post-hoc explanation methods attempt to make the inner workings of deep ...
research
03/02/2023

A Notion of Feature Importance by Decorrelation and Detection of Trends by Random Forest Regression

In many studies, we want to determine the influence of certain features ...
research
10/01/2019

Randomized Ablation Feature Importance

Given a model f that predicts a target y from a vector of input features...
research
09/30/2022

Evaluation of importance estimators in deep learning classifiers for Computed Tomography

Deep learning has shown superb performance in detecting objects and clas...
research
10/15/2021

Evaluating the Faithfulness of Importance Measures in NLP by Recursively Masking Allegedly Important Tokens and Retraining

To explain NLP models, many methods inform which inputs tokens are impor...
research
04/26/2023

On Pitfalls of RemOve-And-Retrain: Data Processing Inequality Perspective

Approaches for appraising feature importance approximations, alternative...

Please sign up or login with your details

Forgot password? Click here to reset