Explaining the Road Not Taken

03/27/2021
by   Hua Shen, et al.
0

It is unclear if existing interpretations of deep neural network models respond effectively to the needs of users. This paper summarizes the common forms of explanations (such as feature attribution, decision rules, or probes) used in over 200 recent papers about natural language processing (NLP), and compares them against user questions collected in the XAI Question Bank. We found that although users are interested in explanations for the road not taken – namely, why the model chose one result and not a well-defined, seemly similar legitimate counterpart – most model interpretations cannot answer these questions.

READ FULL TEXT
research
08/13/2018

Learning Explanations from Language Data

PatternAttribution is a recent method, introduced in the vision domain, ...
research
08/26/2020

How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labels

Explaining to users why automated systems make certain mistakes is impor...
research
10/04/2020

Explaining Deep Neural Networks

Deep neural networks are becoming more and more popular due to their rev...
research
06/01/2020

Aligning Faithful Interpretations with their Social Attribution

We find that the requirement of model interpretations to be faithful is ...
research
05/23/2022

A Question-Answer Driven Approach to Reveal Affirmative Interpretations from Verbal Negations

This paper explores a question-answer driven approach to reveal affirmat...
research
04/09/2021

Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks

Explaining neural network models is important for increasing their trust...
research
04/01/2022

Extracting Rules from Neural Networks with Partial Interpretations

We investigate the problem of extracting rules, expressed in Horn logic,...

Please sign up or login with your details

Forgot password? Click here to reset