Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAy

06/24/2021
by   Christopher J. Anders, et al.
20

Deep Neural Networks (DNNs) are known to be strong predictors, but their prediction strategies can rarely be understood. With recent advances in Explainable Artificial Intelligence, approaches are available to explore the reasoning behind those complex models' predictions. One class of approaches are post-hoc attribution methods, among which Layer-wise Relevance Propagation (LRP) shows high performance. However, the attempt at understanding a DNN's reasoning often stops at the attributions obtained for individual samples in input space, leaving the potential for deeper quantitative analyses untouched. As a manual analysis without the right tools is often unnecessarily labor intensive, we introduce three software packages targeted at scientists to explore model reasoning using attribution approaches and beyond: (1) Zennit - a highly customizable and intuitive attribution framework implementing LRP and related approaches in PyTorch, (2) CoRelAy - a framework to easily and quickly construct quantitative analysis pipelines for dataset-wide analyses of explanations, and (3) ViRelAy - a web-application to interactively explore data, attributions, and analysis results.

READ FULL TEXT

page 4

page 6

06/07/2022

From "Where" to "What": Towards Human-Understandable Explanations through Concept Relevance Propagation

The emerging field of eXplainable Artificial Intelligence (XAI) aims to ...
02/06/2019

Global Explanations of Neural Networks: Mapping the Landscape of Predictions

A barrier to the wider adoption of neural networks is their lack of inte...
11/08/2018

Looking Deeper into Deep Learning Model: Attribution-based Explanations of TextCNN

Layer-wise Relevance Propagation (LRP) and saliency maps have been recen...
12/29/2021

Towards a Shapley Value Graph Framework for Medical peer-influence

eXplainable Artificial Intelligence (XAI) is a sub-field of Artificial I...
01/18/2021

Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbation

With the wide use of deep neural networks (DNN), model interpretability ...
03/18/2021

Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset

Despite the increasingly successful application of neural networks to ma...
07/19/2021

Improving Interpretability of Deep Neural Networks in Medical Diagnosis by Investigating the Individual Units

As interpretability has been pointed out as the obstacle to the adoption...

Code Repositories

zennit

Zennit is a high-level framework in Python using PyTorch for explaining/exploring neural networks using attribution methods like LRP.


view repo

virelay

ViRelAy is a visualization tool for the analysis of data as generated by CoRelAy.


view repo

corelay

CoRelAy is a tool to compose small-scale (single-machine) analysis pipelines.


view repo