Learning Important Features Through Propagating Activation Differences

04/10/2017

∙

The purported "black box"' nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. By optionally giving separate consideration to positive and negative contributions, DeepLIFT can also reveal dependencies which are missed by other approaches. Scores can be computed efficiently in a single backward pass. We apply DeepLIFT to models trained on MNIST and simulated genomic data, and show significant advantages over gradient-based methods. A detailed video tutorial on the method is at http://goo.gl/qKb7pL and code is at http://goo.gl/RM8jvH.

READ FULL TEXT

Learning Important Features Through Propagating Activation Differences

Not Just a Black Box: Learning Important Features Through Propagating Activation Differences

Explaining Neural Networks via Perturbing Important Learned Features

Using Linear Regression for Iteratively Training Neural Networks

Explainable GeoAI: Can saliency maps help interpret artificial intelligence's learning process? An empirical study on natural feature detection

Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks

Prototype-based interpretation of the functionality of neurons in winner-take-all neural networks

Condition Integration Memory Network: An Interpretation of the Meaning of the Neuronal Design

Learning Important Features Through Propagating Activation Differences

Related Research

Not Just a Black Box: Learning Important Features Through Propagating Activation Differences

Explaining Neural Networks via Perturbing Important Learned Features

Using Linear Regression for Iteratively Training Neural Networks

Explainable GeoAI: Can saliency maps help interpret artificial intelligence's learning process? An empirical study on natural feature detection

Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks

Prototype-based interpretation of the functionality of neurons in winner-take-all neural networks

Condition Integration Memory Network: An Interpretation of the Meaning of the Neuronal Design