Cause and Effect: Concept-based Explanation of Neural Networks

05/14/2021
by   Mohammad Nokhbeh Zaeem, et al.
156

In many scenarios, human decisions are explained based on some high-level concepts. In this work, we take a step in the interpretability of neural networks by examining their internal representation or neuron's activations against concepts. A concept is characterized by a set of samples that have specific features in common. We propose a framework to check the existence of a causal relationship between a concept (or its negation) and task classes. While the previous methods focus on the importance of a concept to a task class, we go further and introduce four measures to quantitatively determine the order of causality. Through experiments, we demonstrate the effectiveness of the proposed method in explaining the relationship between a concept and the predictive behaviour of a neural network.

READ FULL TEXT

page 6

page 8

page 9

page 11

page 12

page 13

research
10/17/2019

On Concept-Based Explanations in Deep Neural Networks

Deep neural networks (DNNs) build high-level intelligence on low-level r...
research
12/23/2020

Analyzing Representations inside Convolutional Neural Networks

How can we discover and succinctly summarize the concepts that a neural ...
research
02/05/2020

CHAIN: Concept-harmonized Hierarchical Inference Interpretation of Deep Convolutional Neural Networks

With the great success of networks, it witnesses the increasing demand f...
research
01/10/2018

Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks

In an effort to understand the meaning of the intermediate representatio...
research
06/09/2022

Spatial-temporal Concept based Explanation of 3D ConvNets

Recent studies have achieved outstanding success in explaining 2D image ...
research
05/01/2021

A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts

Despite substantial progress in applying neural networks (NN) to a wide ...
research
04/14/2021

An Interpretability Illusion for BERT

We describe an "interpretability illusion" that arises when analyzing th...

Please sign up or login with your details

Forgot password? Click here to reset