Log In Sign Up

Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks

by   João Loula, et al.

Systematic compositionality is the ability to recombine meaningful units with regular and predictable outcomes, and it's seen as key to humans' capacity for generalization in language. Recent work has studied systematic compositionality in modern seq2seq models using generalization to novel navigation instructions in a grounded environment as a probing tool, requiring models to quickly bootstrap the meaning of new words. We extend this framework here to settings where the model needs only to recombine well-trained functional words (such as "around" and "right") in novel contexts. Our findings confirm and strengthen the earlier ones: seq2seq models can be impressively good at generalizing to novel combinations of previously-seen input, but only when they receive extensive training on the specific pattern to be generalized (e.g., generalizing from many examples of "X around right" to "jump around right"), while failing when generalization requires novel application of compositional rules (e.g., inferring the meaning of "around right" from those of "right" and "around").


A Benchmark for Systematic Generalization in Grounded Language Understanding

Human language users easily interpret expressions that describe unfamili...

COGS: A Compositional Generalization Challenge Based on Semantic Interpretation

Natural language is characterized by compositionality: the meaning of a ...

Improving Systematic Generalization Through Modularity and Augmentation

Systematic generalization is the ability to combine known parts into nov...

Compositional Networks Enable Systematic Generalization for Grounded Language Understanding

Humans are remarkably flexible when understanding new sentences that inc...

Compositional Generalization in Grounded Language Learning via Induced Model Sparsity

We provide a study of how induced model sparsity can help achieve compos...

Emergent Systematic Generalization in a Situated Agent

The question of whether deep neural networks are good at generalising be...