Why should you trust my interpretation? Understanding uncertainty in LIME predictions

04/29/2019
by   Hui Fen, et al.
0

Methods for interpreting machine learning black-box models increase the outcomes' transparency and in turn generates insight into the reliability and fairness of the algorithms. However, the interpretations themselves could contain significant uncertainty that undermines the trust in the outcomes and raises concern about the model's reliability. Focusing on the method "Local Interpretable Model-agnostic Explanations" (LIME), we demonstrate the presence of two sources of uncertainty, namely the randomness in its sampling procedure and the variation of interpretation quality across different input data points. Such uncertainty is present even in models with high training and test accuracy. We apply LIME to synthetic data and two public data sets, text classification in 20 Newsgroup and recidivism risk-scoring in COMPAS, to support our argument.

READ FULL TEXT

page 4

page 7

page 8

page 9

research
12/15/2022

Explainable Machine Learning for Hydrocarbon Prospect Risking

Hydrocarbon prospect risking is a critical application in geophysics pre...
research
05/18/2023

BELLA: Black box model Explanations by Local Linear Approximations

In recent years, understanding the decision-making process of black-box ...
research
06/22/2017

MAGIX: Model Agnostic Globally Interpretable Explanations

Explaining the behavior of a black box machine learning model at the ins...
research
11/22/2016

An unexpected unity among methods for interpreting model predictions

Understanding why a model made a certain prediction is crucial in many d...
research
08/29/2019

Human-grounded Evaluations of Explanation Methods for Text Classification

Due to the black-box nature of deep learning models, methods for explain...
research
11/17/2020

Learning outside the Black-Box: The pursuit of interpretable models

Machine Learning has proved its ability to produce accurate models but t...
research
10/01/2021

Consistent Explanations by Contrastive Learning

Understanding and explaining the decisions of neural networks are critic...

Please sign up or login with your details

Forgot password? Click here to reset