Interpretation of Deep Temporal Representations by Selective Visualization of Internally Activated Units

04/27/2020
by   Sohee Cho, et al.
6

Recently deep neural networks demonstrate competitive performances in classification and regression tasks for many temporal or sequential data. However, it is still hard to understand the classification mechanisms of temporal deep neural networks. In this paper, we propose two new frameworks to visualize temporal representations learned from deep neural networks. Given input data and output, our algorithm interprets the decision of temporal neural network by extracting highly activated periods and visualizes a sub-sequence of input data which contributes to activate the units. Furthermore, we characterize such sub-sequences with clustering and calculate the uncertainty of the suggested type and actual data. We also suggest Layer-wise Relevance from the output of a unit, not from the final output, with backward Monte-Carlo dropout to show the relevance scores of each input point to activate units with providing a visual representation of the uncertainty about this impact.

READ FULL TEXT

page 8

page 13

page 14

page 15

page 16

page 17

page 18

research
08/12/2016

Deep Motif Dashboard: Visualizing and Understanding Genomic Sequences Using Deep Neural Networks

Deep neural network (DNN) models have recently obtained state-of-the-art...
research
08/04/2020

On Feature Relevance Uncertainty: A Monte Carlo Dropout Sampling Approach

Understanding decisions made by neural networks is key for the deploymen...
research
04/04/2016

Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers

Layer-wise relevance propagation is a framework which allows to decompos...
research
12/10/2019

Removable and/or Repeated Units Emerge in Overparametrized Deep Neural Networks

Deep neural networks (DNNs) perform well on a variety of tasks despite t...
research
07/09/2018

Interpreting and Explaining Deep Neural Networks for Classification of Audio Signals

Interpretability of deep neural networks is a recently emerging area of ...
research
12/21/2013

Intriguing properties of neural networks

Deep neural networks are highly expressive models that have recently ach...
research
07/02/2020

Are there any 'object detectors' in the hidden layers of CNNs trained to identify objects or scenes?

Various methods of measuring unit selectivity have been developed with t...

Please sign up or login with your details

Forgot password? Click here to reset