Discovery of Natural Language Concepts in Individual Units of CNNs

02/18/2019
by   Seil Na, et al.
0

Although deep convolutional networks have achieved improved performance in many natural language tasks, they have been treated as black boxes because they are difficult to interpret. Especially, little is known about how they represent language in their intermediate layers. In an attempt to understand the representations of deep convolutional networks trained on language tasks, we show that individual units are selectively responsive to specific morphemes, words, and phrases, rather than responding to arbitrary and uninterpretable patterns. In order to quantitatively analyze such an intriguing phenomenon, we propose a concept alignment method based on how units respond to the replicated text. We conduct analyses with different architectures on multiple datasets for classification and translation tasks and provide new insights into how deep models understand natural language.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/15/2017

Interpreting Deep Visual Representations via Network Dissection

The success of recent deep convolutional neural networks (CNNs) depends ...
research
10/06/2017

Deep Convolutional Neural Networks as Generic Feature Extractors

Recognizing objects in natural images is an intricate problem involving ...
research
07/10/2019

Neural Networks as Explicit Word-Based Rules

Filters of convolutional networks used in computer vision are often visu...
research
04/19/2017

Network Dissection: Quantifying Interpretability of Deep Visual Representations

We propose a general framework called Network Dissection for quantifying...
research
08/12/2018

Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference

Deep learning models have achieved remarkable success in natural languag...
research
07/17/2019

Differentiable Disentanglement Filter: an Application Agnostic Core Concept Discovery Probe

It has long been speculated that deep neural networks function by discov...
research
10/19/2018

Understanding Deep Convolutional Networks through Gestalt Theory

The superior performance of deep convolutional networks over high-dimens...

Please sign up or login with your details

Forgot password? Click here to reset