Understanding intermediate layers using linear classifier probes

10/05/2016
by   Guillaume Alain, et al.
0

Neural network models have a reputation for being black boxes. We propose a new method to understand better the roles and dynamics of the intermediate layers. This has direct consequences on the design of such models and it enables the expert to be able to justify certain heuristics (such as the auxiliary heads in the Inception model). Our method uses linear classifiers, referred to as "probes", where a probe can only use the hidden units of a given intermediate layer as discriminating features. Moreover, these probes cannot affect the training phase of a model, and they are generally added after training. They allow the user to visualize the state of the model at multiple steps of training. We demonstrate how this can be used to develop a better intuition about a known model and to diagnose potential problems.

READ FULL TEXT

page 3

page 5

page 7

research
05/14/2021

Sparsity-Probe: Analysis tool for Deep Learning Models

We propose a probe for the analysis of deep learning architectures that ...
research
03/12/2023

Interpreting Hidden Semantics in the Intermediate Layers of 3D Point Cloud Classification Neural Network

Although 3D point cloud classification neural network models have been w...
research
06/27/2017

DE-PACRR: Exploring Layers Inside the PACRR Model

Recent neural IR models have demonstrated deep learning's utility in ad-...
research
07/19/2018

Improving Simple Models with Confidence Profiles

In this paper, we propose a new method called ProfWeight for transferrin...
research
12/16/2021

Self-Supervised Learning for speech recognition with Intermediate layer supervision

Recently, pioneer work finds that speech pre-trained models can solve fu...
research
05/14/2018

Confidence Scoring Using Whitebox Meta-models with Linear Classifier Probes

We propose a confidence scoring mechanism for multi-layer neural network...
research
02/10/2022

Coded ResNeXt: a network for designing disentangled information paths

To avoid treating neural networks as highly complex black boxes, the dee...

Please sign up or login with your details

Forgot password? Click here to reset