Discovering Concepts in Learned Representations using Statistical Inference and Interactive Visualization

02/09/2022
by   Adrianna Janik, et al.
0

Concept discovery is one of the open problems in the interpretability literature that is important for bridging the gap between non-deep learning experts and model end-users. Among current formulations, concepts defines them by as a direction in a learned representation space. This definition makes it possible to evaluate whether a particular concept significantly influences classification decisions for classes of interest. However, finding relevant concepts is tedious, as representation spaces are high-dimensional and hard to navigate. Current approaches include hand-crafting concept datasets and then converting them to latent space directions; alternatively, the process can be automated by clustering the latent space. In this study, we offer another two approaches to guide user discovery of meaningful concepts, one based on multiple hypothesis testing, and another on interactive visualization. We explore the potential value and limitations of these approaches through simulation experiments and an demo visual interface to real data. Overall, we find that these techniques offer a promising strategy for discovering relevant concepts in settings where users do not have predefined descriptions of them, but without completely automating the process.

READ FULL TEXT

page 3

page 5

page 6

page 7

research
04/04/2022

ConceptExplainer: Understanding the Mental Model of Deep Learning Algorithms via Interactive Concept-based Explanations

Traditional deep learning interpretability methods which are suitable fo...
research
05/08/2023

Interactive Concept Learning for Uncovering Latent Themes in Large Text Collections

Experts across diverse disciplines are often interested in making sense ...
research
04/22/2022

SegDiscover: Visual Concept Discovery via Unsupervised Semantic Segmentation

Visual concept discovery has long been deemed important to improve inter...
research
12/04/2021

Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations

Learning visual concepts from raw images without strong supervision is a...
research
07/12/2018

Interactive Discovery System for Direct Democracy

Decide Madrid is the civic technology of Madrid City Council which allow...
research
04/29/2022

Concept Activation Vectors for Generating User-Defined 3D Shapes

We explore the interpretability of 3D geometric deep learning models in ...
research
09/22/2022

Concept Activation Regions: A Generalized Framework For Concept-Based Explanations

Concept-based explanations permit to understand the predictions of a dee...

Please sign up or login with your details

Forgot password? Click here to reset