Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations

11/25/2020
by   Wolfgang Stammer, et al.
17

Most explanation methods in deep learning map importance estimates for a model's prediction back to the original input space. These "visual" explanations are often insufficient, as the model's actual concept remains elusive. Moreover, without insights into the model's semantic concept, it is difficult – if not impossible – to intervene on the model's behavior via its explanations, called Explanatory Interactive Learning. Consequently, we propose to intervene on a Neuro-Symbolic scene representation, which allows one to revise the model on the semantic level, e.g. "never focus on the color to make your decision". We compiled a novel confounded visual scene data set, the CLEVR-Hans data set, capturing complex compositions of different objects. The results of our experiments on CLEVR-Hans demonstrate that our semantic explanations, i.e. compositional explanations at a per-object level, can identify confounders that are not identifiable using "visual" explanations only. More importantly, feedback on this semantic level makes it possible to revise the model from focusing on these confounding factors.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 12

page 15

02/25/2022

Human-Centered Concept Explanations for Neural Networks

Understanding complex machine learning models such as deep neural networ...
11/15/2021

LIMEcraft: Handcrafted superpixel selection and inspection for Visual eXplanations

The increased interest in deep learning applications, and their hard-to-...
04/04/2022

ConceptExplainer: Understanding the Mental Model of Deep Learning Algorithms via Interactive Concept-based Explanations

Traditional deep learning interpretability methods which are suitable fo...
07/20/2022

Overlooked factors in concept-based explanations: Dataset choice, concept salience, and human capability

Concept-based interpretability methods aim to explain deep neural networ...
06/23/2022

A Model-Agnostic SAT-based Approach for Symbolic Explanation Enumeration

In this paper titled A Model-Agnostic SAT-based approach for Symbolic Ex...
08/26/2022

Concept-Based Techniques for "Musicologist-friendly" Explanations in a Deep Music Classifier

Current approaches for explaining deep learning systems applied to music...
05/24/2021

Deep Descriptive Clustering

Recent work on explainable clustering allows describing clusters when th...

Code Repositories

CLEVR-Hans

Repository for the CLEVR-Hans dataset


view repo