DeepAI AI Chat
Log In Sign Up

Investigating sanity checks for saliency maps with image and text classification

by   Narine Kokhlikyan, et al.

Saliency maps have shown to be both useful and misleading for explaining model predictions especially in the context of images. In this paper, we perform sanity checks for text modality and show that the conclusions made for image do not directly transfer to text. We also analyze the effects of the input multiplier in certain saliency maps using similarity scores, max-sensitivity and infidelity evaluation metrics. Our observations reveal that the input multiplier carries input's structural patterns in explanation maps, thus leading to similar results regardless of the choice of model parameters. We also show that the smoothness of a Neural Network (NN) function can affect the quality of saliency-based explanations. Our investigations reveal that replacing ReLUs with Softplus and MaxPool with smoother variants such as LogSumExp (LSE) can lead to explanations that are more reliable based on the infidelity evaluation metric.


page 1

page 2

page 3

page 4


Evaluating Input Perturbation Methods for Interpreting CNNs and Saliency Map Comparison

Input perturbation methods occlude parts of an input to a function and m...

Learning a Saliency Evaluation Metric Using Crowdsourced Perceptual Judgments

In the area of human fixation prediction, dozens of computational salien...

Building Reliable Explanations of Unreliable Neural Networks: Locally Smoothing Perspective of Model Interpretation

We present a novel method for reliably explaining the predictions of neu...

Saliency Maps Generation for Automatic Text Summarization

Saliency map generation techniques are at the forefront of explainable A...

Where do Models go Wrong? Parameter-Space Saliency Maps for Explainability

Conventional saliency maps highlight input features to which neural netw...

Interpreting BERT-based Text Similarity via Activation and Saliency Maps

Recently, there has been growing interest in the ability of Transformer-...

Evaluating the Use of Reconstruction Error for Novelty Localization

The pixelwise reconstruction error of deep autoencoders is often utilize...