DeepAI AI Chat
Log In Sign Up

Compositionality as Lexical Symmetry

by   Ekin Akyürek, et al.

Standard deep network models lack the inductive biases needed to generalize compositionally in tasks like semantic parsing, translation, and question answering. A large body of work in natural language processing seeks to overcome this limitation with new model architectures that enforce a compositional process of sentence interpretation. In this paper, we present a domain-general framework for compositional modeling that instead formulates compositionality as a constraint on data distributions. We prove that for any task factorizable into a lexicon and a composition function, there exists a family of data transformation functions that are guaranteed to produce new, well-formed examples when applied to training data. We further show that it is possible to identify these data transformations even when the composition function is unknown (e.g. when we do not know how to write or infer a symbolic grammar). Using these transformation functions to perform data augmentation for ordinary RNN and transformer sequence models, we obtain state-of-the-art results on the CLEVR-CoGenT visual question answering dataset, and results comparable to specialized model architectures on the COGS semantic parsing dataset.


page 6

page 8


Evaluating Semantic Parsing against a Simple Web-based Question Answering Model

Semantic parsing shines at analyzing complex natural language that invol...

Making Transformers Solve Compositional Tasks

Several studies have reported the inability of Transformer models to gen...

SUBS: Subtree Substitution for Compositional Semantic Parsing

Although sequence-to-sequence models often achieve good performance in s...

Compositional Task-Oriented Parsing as Abstractive Question Answering

Task-oriented parsing (TOP) aims to convert natural language into machin...

Compositional pre-training for neural semantic parsing

Semantic parsing is the process of translating natural language utteranc...

Neural Compositional Denotational Semantics for Question Answering

Answering compositional questions requiring multi-step reasoning is chal...

Routing Networks and the Challenges of Modular and Compositional Computation

Compositionality is a key strategy for addressing combinatorial complexi...