DeepAI AI Chat
Log In Sign Up

On the Veracity of Local, Model-agnostic Explanations in Audio Classification: Targeted Investigations with Adversarial Examples

by   Verena Praher, et al.
Johannes Kepler University Linz

Local explanation methods such as LIME have become popular in MIR as tools for generating post-hoc, model-agnostic explanations of a model's classification decisions. The basic idea is to identify a small set of human-understandable features of the classified example that are most influential on the classifier's prediction. These are then presented as an explanation. Evaluation of such explanations in publications often resorts to accepting what matches the expectation of a human without actually being able to verify if what the explanation shows is what really caused the model's prediction. This paper reports on targeted investigations where we try to get more insight into the actual veracity of LIME's explanations in an audio classification task. We deliberately design adversarial examples for the classifier, in a way that gives us knowledge about which parts of the input are potentially responsible for the model's (wrong) prediction. Asking LIME to explain the predictions for these adversaries permits us to study whether local explanations do indeed detect these regions of interest. We also look at whether LIME is more successful in finding perturbations that are more prominent and easily noticeable for a human. Our results suggest that LIME does not necessarily manage to identify the most relevant input features and hence it remains unclear whether explanations are useful or even misleading.


EDDA: Explanation-driven Data Augmentation to Improve Model and Explanation Alignment

Recent years have seen the introduction of a range of methods for post-h...

Reliable Local Explanations for Machine Listening

One way to analyse the behaviour of machine learning models is through l...

Debugging Tests for Model Explanations

We investigate whether post-hoc model explanations are effective for dia...

Why is the prediction wrong? Towards underfitting case explanation via meta-classification

In this paper we present a heuristic method to provide individual explan...

Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations

In attempts to "explain" predictions of machine learning models, researc...

Deja vu from the SVM Era: Example-based Explanations with Outlier Detection

Understanding the features that contributed to a prediction is important...

A Model-Agnostic SAT-based Approach for Symbolic Explanation Enumeration

In this paper titled A Model-Agnostic SAT-based approach for Symbolic Ex...

Code Repositories