DeepAI AI Chat
Log In Sign Up

Investigating sanity checks for saliency maps with image and text classification

06/08/2021
by   Narine Kokhlikyan, et al.
Facebook
0

Saliency maps have shown to be both useful and misleading for explaining model predictions especially in the context of images. In this paper, we perform sanity checks for text modality and show that the conclusions made for image do not directly transfer to text. We also analyze the effects of the input multiplier in certain saliency maps using similarity scores, max-sensitivity and infidelity evaluation metrics. Our observations reveal that the input multiplier carries input's structural patterns in explanation maps, thus leading to similar results regardless of the choice of model parameters. We also show that the smoothness of a Neural Network (NN) function can affect the quality of saliency-based explanations. Our investigations reveal that replacing ReLUs with Softplus and MaxPool with smoother variants such as LogSumExp (LSE) can lead to explanations that are more reliable based on the infidelity evaluation metric.

READ FULL TEXT

page 1

page 2

page 3

page 4

01/26/2021

Evaluating Input Perturbation Methods for Interpreting CNNs and Saliency Map Comparison

Input perturbation methods occlude parts of an input to a function and m...
06/27/2018

Learning a Saliency Evaluation Metric Using Crowdsourced Perceptual Judgments

In the area of human fixation prediction, dozens of computational salien...
03/26/2021

Building Reliable Explanations of Unreliable Neural Networks: Locally Smoothing Perspective of Model Interpretation

We present a novel method for reliably explaining the predictions of neu...
07/12/2019

Saliency Maps Generation for Automatic Text Summarization

Saliency map generation techniques are at the forefront of explainable A...
08/03/2021

Where do Models go Wrong? Parameter-Space Saliency Maps for Explainability

Conventional saliency maps highlight input features to which neural netw...
08/13/2022

Interpreting BERT-based Text Similarity via Activation and Saliency Maps

Recently, there has been growing interest in the ability of Transformer-...
07/28/2021

Evaluating the Use of Reconstruction Error for Novelty Localization

The pixelwise reconstruction error of deep autoencoders is often utilize...