Compositional Generalization Requires Compositional Parsers

by   Pia Weißenhorn, et al.

A rapidly growing body of research on compositional generalization investigates the ability of a semantic parser to dynamically recombine linguistic elements seen in training into unseen sequences. We present a systematic comparison of sequence-to-sequence models and models guided by compositional principles on the recent COGS corpus (Kim and Linzen, 2020). Though seq2seq models can perform well on lexical tasks, they perform with near-zero accuracy on structural generalization tasks that require novel syntactic structures; this holds true even when they are trained to predict syntax instead of semantics. In contrast, compositional models achieve near-perfect accuracy on structural generalization; we present new results confirming this from the AM parser (Groschwitz et al., 2021). Our findings show structural generalization is a key measure of compositional generalization and requires models that are aware of complex structure.


page 1

page 2

page 3

page 4


Structural generalization is hard for sequence-to-sequence models

Sequence-to-sequence (seq2seq) models have been successful across many N...

Learning Algebraic Recombination for Compositional Generalization

Neural sequence models exhibit limited compositional generalization abil...

Modelling structural zeros in compositional data via a zero-censored multivariate normal model

We present a new model for analyzing compositional data with structural ...

Compositional Generalisation with Structured Reordering and Fertility Layers

Seq2seq models have been shown to struggle with compositional generalisa...

COGS: A Compositional Generalization Challenge Based on Semantic Interpretation

Natural language is characterized by compositionality: the meaning of a ...

Revisiting the Compositional Generalization Abilities of Neural Sequence Models

Compositional generalization is a fundamental trait in humans, allowing ...

Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models

Human linguistic capacity is often characterized by compositionality and...

Please sign up or login with your details

Forgot password? Click here to reset