DeepAI AI Chat
Log In Sign Up

Jump to better conclusions: SCAN both left and right

09/12/2018
by   Joost Bastings, et al.
Facebook
University of Amsterdam
0

Lake and Baroni (2018) recently introduced the SCAN data set, which consists of simple commands paired with action sequences and is intended to test the strong generalization abilities of recurrent sequence-to-sequence models. Their initial experiments suggested that such models may fail because they lack the ability to extract systematic rules. Here, we take a closer look at SCAN and show that it does not always capture the kind of generalization that it was designed for. To mitigate this we propose a complementary dataset, which requires mapping actions back to the original commands, called NACS. We show that models that do well on SCAN do not necessarily do well on NACS, and that NACS exhibits properties more closely aligned with realistic use-cases for sequence-to-sequence models.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/12/2019

Compositional generalization through meta sequence-to-sequence learning

People can learn a new concept and use it compositionally, understanding...
09/02/2021

Sequence-to-Sequence Learning with Latent Neural Grammars

Sequence-to-sequence learning with neural networks has become the de fac...
03/14/2020

Synonymous Generalization in Sequence-to-Sequence Recurrent Networks

When learning a language, people can quickly expand their understanding ...
11/28/2022

Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionality

Recent datasets expose the lack of the systematic generalization ability...
10/30/2022

Real-Time MRI Video synthesis from time aligned phonemes with sequence-to-sequence networks

Real-Time Magnetic resonance imaging (rtMRI) of the midsagittal plane of...