DeepAI AI Chat
Log In Sign Up

Neuron ranking -- an informed way to condense convolutional neural networks architecture

by   Kamil Adamczewski, et al.
Max Planck Society

Convolutional neural networks (CNNs) in recent years have made a dramatic impact in science, technology and industry, yet the theoretical mechanism of CNN architecture design remains surprisingly vague. The CNN neurons, including its distinctive element, convolutional filters, are known to be learnable features, yet their individual role in producing the output is rather unclear. The thesis of this work is that not all neurons are equally important and some of them contain more useful information to perform a given task . Consequently, we quantify the significance of each filter and rank its importance in describing input to produce the desired output. This work presents two different methods: (1) a game theoretical approach based on Shapley value which computes the marginal contribution of each filter; and (2) a probabilistic approach based on what-we-call, the importance switch using variational inference. Strikingly, these two vastly different methods produce similar experimental results, confirming the general theory that some of the filters are inherently more important that the others. The learned ranks can be readily useable for network compression and interpretability.


page 8

page 12

page 13


Interpreting Convolutional Neural Networks Through Compression

Convolutional neural networks (CNNs) achieve state-of-the-art performanc...

A rotation-equivariant convolutional neural network model of primary visual cortex

Classical models describe primary visual cortex (V1) as a filter bank of...

NISP: Pruning Networks using Neuron Importance Score Propagation

To reduce the significant redundancy in deep Convolutional Neural Networ...

PCACE: A Statistical Approach to Ranking Neurons for CNN Interpretability

In this paper we introduce a new problem within the growing literature o...

Neural system identification for large populations separating "what" and "where"

Neuroscientists classify neurons into different types that perform simil...

Basic Filters for Convolutional Neural Networks Applied to Music: Training or Design?

When convolutional neural networks are used to tackle learning problems ...