Explaining Character-Aware Neural Networks for Word-Level Prediction: Do They Discover Linguistic Rules?

08/28/2018
by   Fréderic Godin, et al.
0

Character-level features are currently used in different neural network-based natural language processing algorithms. However, little is known about the character-level patterns those models learn. Moreover, models are often compared only quantitatively while a qualitative analysis is missing. In this paper, we investigate which character-level patterns neural networks learn and if those patterns coincide with manually-defined word segmentations and annotations. To that end, we extend the contextual decomposition technique (Murdoch et al. 2018) to convolutional neural networks which allows us to compare convolutional neural networks and bidirectional long short-term memory networks. We evaluate and compare these models for the task of morphological tagging on three morphologically different languages and show that these models implicitly discover understandable linguistic rules. Our implementation can be found at https://github.com/FredericGodin/ContextualDecomposition-NLP .

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

12/23/2017

Dual Long Short-Term Memory Networks for Sub-Character Representation Learning

Characters have commonly been regarded as the minimal processing unit in...
01/19/2018

Evaluating neural network explanation methods using hybrid documents and morphological prediction

We propose two novel paradigms for evaluating neural network explanation...
03/07/2021

TypeShift: A User Interface for Visualizing the Typing Production Process

TypeShift is a tool for visualizing linguistic patterns in the timing of...
11/17/2021

Character Transformations for Non-Autoregressive GEC Tagging

We propose a character-based nonautoregressive GEC approach, with automa...
03/15/2017

Character-based Neural Embeddings for Tweet Clustering

In this paper we show how the performance of tweet clustering can be imp...
03/10/2021

Relational Weight Priors in Neural Networks for Abstract Pattern Learning and Language Modelling

Deep neural networks have become the dominant approach in natural langua...
01/18/2022

Dilated Convolutional Neural Networks for Lightweight Diacritics Restoration

Diacritics restoration has become a ubiquitous task in the Latin-alphabe...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.