Do RNN States Encode Abstract Phonological Processes?

04/01/2021
by   Miikka Silfverberg, et al.
0

Sequence-to-sequence models have delivered impressive results in word formation tasks such as morphological inflection, often learning to model subtle morphophonological details with limited training data. Despite the performance, the opacity of neural models makes it difficult to determine whether complex generalizations are learned, or whether a kind of separate rote memorization of each morphophonological process takes place. To investigate whether complex alternations are simply memorized or whether there is some level of generalization across related sound changes in a sequence-to-sequence model, we perform several experiments on Finnish consonant gradation – a complex set of sound changes triggered in some words by certain suffixes. We find that our models often – though not always – encode 17 different consonant gradation processes in a handful of dimensions in the RNN. We also show that by scaling the activations in these dimensions we can control whether consonant gradation occurs and the direction of the gradation.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

05/21/2018

MorphNet: A sequence-to-sequence model that combines morphological analysis and disambiguation

We introduce MorphNet, a single model that combines morphological analys...
02/16/2021

Searching for Search Errors in Neural Morphological Inflection

Neural sequence-to-sequence models are currently the predominant choice ...
04/25/2018

Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models

Neural Sequence-to-Sequence models have proven to be accurate and robust...
10/24/2020

Neural Compound-Word (Sandhi) Generation and Splitting in Sanskrit Language

This paper describes neural network based approaches to the process of t...
02/05/2019

Model Unit Exploration for Sequence-to-Sequence Speech Recognition

We evaluate attention-based encoder-decoder models along two dimensions:...
05/20/2018

Learning compositionally through attentive guidance

In this paper, we introduce Attentive Guidance (AG), a new mechanism to ...
08/22/2019

The compositionality of neural networks: integrating symbolism and connectionism

Despite a multitude of empirical studies, little consensus exists on whe...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.