Explaining Classifiers by Constructing Familiar Concepts

03/07/2022
by   Johannes Schneider, et al.
55

Interpreting a large number of neurons in deep learning is difficult. Our proposed `CLAssifier-DECoder' architecture (ClaDec) facilitates the understanding of the output of an arbitrary layer of neurons or subsets thereof. It uses a decoder that transforms the incomprehensible representation of the given neurons to a representation that is more similar to the domain a human is familiar with. In an image recognition problem, one can recognize what information (or concepts) a layer maintains by contrasting reconstructed images of ClaDec with those of a conventional auto-encoder(AE) serving as reference. An extension of ClaDec allows trading comprehensibility and fidelity. We evaluate our approach for image classification using convolutional neural networks. We show that reconstructed visualizations using encodings from a classifier capture more relevant classification information than conventional AEs. This holds although AEs contain more information on the original input. Our user study highlights that even non-experts can identify a diverse set of concepts contained in images that are relevant (or irrelevant) for the classifier. We also compare against saliency based methods that focus on pixel relevance rather than concepts. We show that ClaDec tends to highlight more relevant input areas to classification though outcomes depend on classifier architecture. Code is at <https://github.com/JohnTailor/ClaDec>

READ FULL TEXT

page 13

page 14

page 15

page 16

page 19

page 20

page 22

page 25

research
05/27/2020

Explaining Neural Networks by Decoding Layer Activations

To derive explanations for deep learning models, ie. classifiers, we pro...
research
12/18/2018

Interactive Naming for Explaining Deep Neural Networks: A Formative Study

We consider the problem of explaining the decisions of deep neural netwo...
research
10/01/2019

Research on insect pest image detection and recognition based on bio-inspired methods

Insect pest recognition is necessary for crop protection in many areas o...
research
03/16/2020

Synthesizing human-like sketches from natural images using a conditional convolutional decoder

Humans are able to precisely communicate diverse concepts by employing s...
research
03/27/2022

HINT: Hierarchical Neuron Concept Explainer

To interpret deep networks, one main approach is to associate neurons wi...
research
01/15/2020

Relevance Prediction from Eye-movements Using Semi-interpretable Convolutional Neural Networks

We propose an image-classification method to predict the perceived-relev...
research
03/05/2018

Style Memory: Making a Classifier Network Generative

Deep networks have shown great performance in classification tasks. Howe...

Please sign up or login with your details

Forgot password? Click here to reset