Interpreting Deep Neural Networks Through Variable Importance

01/28/2019
by   Jonathan Ish-Horowicz, et al.
0

While the success of deep neural networks (DNNs) is well-established across a variety of domains, our ability to explain and interpret these methods is limited. Unlike previously proposed local methods which try to explain particular classification decisions, we focus on global interpretability and ask a universally applicable question: given a trained model, which features are the most important? In the context of neural networks, a feature is rarely important on its own, so our strategy is specifically designed to leverage partial covariance structures and incorporate variable dependence into feature ranking. Our methodological contributions in this paper are two-fold. First, we propose an effect size analogue for DNNs that is appropriate for applications with highly collinear predictors (ubiquitous in computer vision). Second, we extend the recently proposed "RelATive cEntrality" (RATE) measure (Crawford et al., 2019) to the Bayesian deep learning setting. RATE applies an information theoretic criterion to the posterior distribution of effect sizes to assess feature significance. We apply our framework to three broad application areas: computer vision, natural language processing, and social science.

READ FULL TEXT

page 7

page 14

research
03/21/2018

Information Theoretic Interpretation of Deep learning

We interpret part of the experimental results of Shwartz-Ziv and Tishby ...
research
06/18/2021

Towards interpreting computer vision based on transformation invariant optimization

Interpreting how does deep neural networks (DNNs) make predictions is a ...
research
07/01/2020

ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networks

Deep Neural Networks (DNNs) have been applied successfully in computer v...
research
09/23/2020

Deep Neural Networks with Short Circuits for Improved Gradient Learning

Deep neural networks have achieved great success both in computer vision...
research
12/08/2015

Explaining NonLinear Classification Decisions with Deep Taylor Decomposition

Nonlinear methods such as Deep Neural Networks (DNNs) are the gold stand...
research
10/08/2021

Automatic annotation of visual deep neural networks

Computer vision is widely used in the fields of driverless, face recogni...
research
06/29/2023

Seeing in Words: Learning to Classify through Language Bottlenecks

Neural networks for computer vision extract uninterpretable features des...

Please sign up or login with your details

Forgot password? Click here to reset