Interpretation of Black Box NLP Models: A Survey

03/31/2022
by   Shivani Choudhary, et al.
0

An increasing number of machine learning models have been deployed in domains with high stakes such as finance and healthcare. Despite their superior performances, many models are black boxes in nature which are hard to explain. There are growing efforts for researchers to develop methods to interpret these black-box models. Post hoc explanations based on perturbations, such as LIME, are widely used approaches to interpret a machine learning model after it has been built. This class of methods has been shown to exhibit large instability, posing serious challenges to the effectiveness of the method itself and harming user trust. In this paper, we propose S-LIME, which utilizes a hypothesis testing framework based on central limit theorem for determining the number of perturbation points needed to guarantee stability of the resulting explanation. Experiments on both simulated and real world data sets are provided to demonstrate the effectiveness of our method.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/15/2021

S-LIME: Stabilized-LIME for Model Explanation

An increasing number of machine learning models have been deployed in do...
11/12/2020

Robust and Stable Black Box Explanations

As machine learning black boxes are increasingly being deployed in real-...
02/23/2021

Feature Importance Explanations for Temporal Black-Box Models

Models in the supervised learning framework may capture rich and complex...
02/21/2021

Towards the Unification and Robustness of Perturbation and Gradient Based Explanations

As machine learning black boxes are increasingly being deployed in criti...
07/07/2022

An Additive Instance-Wise Approach to Multi-class Model Interpretation

Interpretable machine learning offers insights into what factors drive a...
02/17/2018

Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution

Strong intelligent machines powered by deep neural networks are increasi...
05/29/2019

Hijacking Malaria Simulators with Probabilistic Programming

Epidemiology simulations have become a fundamental tool in the fight aga...