Topological Representations of Local Explanations

by   Peter Xenopoulos, et al.

Local explainability methods – those which seek to generate an explanation for each prediction – are becoming increasingly prevalent due to the need for practitioners to rationalize their model outputs. However, comparing local explainability methods is difficult since they each generate outputs in various scales and dimensions. Furthermore, due to the stochastic nature of some explainability methods, it is possible for different runs of a method to produce contradictory explanations for a given observation. In this paper, we propose a topology-based framework to extract a simplified representation from a set of local explanations. We do so by first modeling the relationship between the explanation space and the model predictions as a scalar function. Then, we compute the topological skeleton of this function. This topological skeleton acts as a signature for such functions, which we use to compare different explanation methods. We demonstrate that our framework can not only reliably identify differences between explainability techniques but also provides stable representations. Then, we show how our framework can be used to identify appropriate parameters for local explainability methods. Our framework is simple, does not require complex optimizations, and can be broadly applied to most local explanation methods. We believe the practicality and versatility of our approach will help promote topology-based approaches as a tool for understanding and comparing explanation methods.


page 5

page 6


Global Counterfactual Explanations: Investigations, Implementations and Improvements

Counterfactual explanations have been widely studied in explainability, ...

The Shape of Explanations: A Topological Account of Rule-Based Explanations in Machine Learning

Rule-based explanations provide simple reasons explaining the behavior o...

The Weighting Game: Evaluating Quality of Explainability Methods

The objective of this paper is to assess the quality of explanation heat...

Explaining by Removing: A Unified Framework for Model Explanation

Researchers have proposed a wide variety of model explanation approaches...

Feature Removal Is a Unifying Principle for Model Explanation Methods

Researchers have proposed a wide variety of model explanation approaches...

Explainability-by-Design: A Methodology to Support Explanations in Decision-Making Systems

Algorithms play a key role nowadays in many technological systems that c...

Model Agnostic Multilevel Explanations

In recent years, post-hoc local instance-level and global dataset-level ...

Please sign up or login with your details

Forgot password? Click here to reset