UFO: A unified method for controlling Understandability and Faithfulness Objectives in concept-based explanations for CNNs

03/27/2023
by   Vikram V. Ramaswamy, et al.
0

Concept-based explanations for convolutional neural networks (CNNs) aim to explain model behavior and outputs using a pre-defined set of semantic concepts (e.g., the model recognizes scene class “bedroom” based on the presence of concepts “bed” and “pillow”). However, they often do not faithfully (i.e., accurately) characterize the model's behavior and can be too complex for people to understand. Further, little is known about how faithful and understandable different explanation methods are, and how to control these two properties. In this work, we propose UFO, a unified method for controlling Understandability and Faithfulness Objectives in concept-based explanations. UFO formalizes understandability and faithfulness as mathematical objectives and unifies most existing concept-based explanations methods for CNNs. Using UFO, we systematically investigate how explanations change as we turn the knobs of faithfulness and understandability. Our experiments demonstrate a faithfulness-vs-understandability tradeoff: increasing understandability reduces faithfulness. We also provide insights into the “disagreement problem” in explainable machine learning, by analyzing when and how concept-based explanations disagree with each other.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/25/2020

Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations

Most explanation methods in deep learning map importance estimates for a...
research
02/25/2022

Human-Centered Concept Explanations for Neural Networks

Understanding complex machine learning models such as deep neural networ...
research
07/20/2022

Overlooked factors in concept-based explanations: Dataset choice, concept salience, and human capability

Concept-based interpretability methods aim to explain deep neural networ...
research
08/11/2023

Scale-Preserving Automatic Concept Extraction (SPACE)

Convolutional Neural Networks (CNN) have become a common choice for indu...
research
12/13/2022

Examining the Difference Among Transformers and CNNs with Explanation Methods

We propose a methodology that systematically applies deep explanation al...
research
05/27/2023

Statistically Significant Concept-based Explanation of Image Classifiers via Model Knockoffs

A concept-based classifier can explain the decision process of a deep le...
research
05/17/2023

Explain Any Concept: Segment Anything Meets Concept-Based Explanation

EXplainable AI (XAI) is an essential topic to improve human understandin...

Please sign up or login with your details

Forgot password? Click here to reset