Input-Cell Attention Reduces Vanishing Saliency of Recurrent Neural Networks

10/27/2019
by   Aya Abdelsalam Ismail, et al.
15

Recent efforts to improve the interpretability of deep neural networks use saliency to characterize the importance of input features to predictions made by models. Work on interpretability using saliency-based methods on Recurrent Neural Networks (RNNs) has mostly targeted language tasks, and their applicability to time series data is less understood. In this work we analyze saliency-based methods for RNNs, both classical and gated cell architectures. We show that RNN saliency vanishes over time, biasing detection of salient features only to later time steps and are, therefore, incapable of reliably detecting important features at arbitrary time intervals. To address this vanishing saliency problem, we propose a novel RNN cell structure (input-cell attention), which can extend any RNN cell architecture. At each time step, instead of only looking at the current input vector, input-cell attention uses a fixed-size matrix embedding, each row of the matrix attending to different inputs from current or previous time steps. Using synthetic data, we show that the saliency map produced by the input-cell attention RNN is able to faithfully detect important features regardless of their occurrence in time. We also apply the input-cell attention RNN on a neuroscience task analyzing functional Magnetic Resonance Imaging (fMRI) data for human subjects performing a variety of tasks. In this case, we use saliency to characterize brain regions (input features) for which activity is important to distinguish between tasks. We show that standard RNN architectures are only capable of detecting important brain regions in the last few time steps of the fMRI data, while the input-cell attention model is able to detect important brain region activity across time without latter time step biases.

READ FULL TEXT

page 2

page 14

page 15

research
10/26/2020

Benchmarking Deep Learning Interpretability in Time Series Predictions

Saliency methods are used extensively to highlight the importance of inp...
research
04/23/2020

Evaluating Adversarial Robustness for Deep Neural Network Interpretability using fMRI Decoding

While deep neural networks (DNNs) are being increasingly used to make pr...
research
10/15/2019

Jointly Discriminative and Generative Recurrent Neural Networks for Learning from fMRI

Recurrent neural networks (RNNs) were designed for dealing with time-ser...
research
08/23/2018

Brain Biomarker Interpretation in ASD Using Deep Learning and fMRI

Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder....
research
11/03/2016

Recurrent Neural Networks for Spatiotemporal Dynamics of Intrinsic Networks from fMRI Data

Functional magnetic resonance imaging (fMRI) of temporally-coherent bloo...
research
01/10/2020

Understanding Graph Isomorphism Network for Brain MR Functional Connectivity Analysis

Graph neural networks (GNN) rely on graph operations that include neural...
research
04/15/2021

Demographic-Guided Attention in Recurrent Neural Networks for Modeling Neuropathophysiological Heterogeneity

Heterogeneous presentation of a neurological disorder suggests potential...

Please sign up or login with your details

Forgot password? Click here to reset