Compositionality as Lexical Symmetry

01/30/2022
by   Ekin Akyürek, et al.
0

Standard deep network models lack the inductive biases needed to generalize compositionally in tasks like semantic parsing, translation, and question answering. A large body of work in natural language processing seeks to overcome this limitation with new model architectures that enforce a compositional process of sentence interpretation. In this paper, we present a domain-general framework for compositional modeling that instead formulates compositionality as a constraint on data distributions. We prove that for any task factorizable into a lexicon and a composition function, there exists a family of data transformation functions that are guaranteed to produce new, well-formed examples when applied to training data. We further show that it is possible to identify these data transformations even when the composition function is unknown (e.g. when we do not know how to write or infer a symbolic grammar). Using these transformation functions to perform data augmentation for ordinary RNN and transformer sequence models, we obtain state-of-the-art results on the CLEVR-CoGenT visual question answering dataset, and results comparable to specialized model architectures on the COGS semantic parsing dataset.

READ FULL TEXT

page 6

page 8

research
07/14/2017

Evaluating Semantic Parsing against a Simple Web-based Question Answering Model

Semantic parsing shines at analyzing complex natural language that invol...
research
08/09/2021

Making Transformers Solve Compositional Tasks

Several studies have reported the inability of Transformer models to gen...
research
05/03/2022

SUBS: Subtree Substitution for Compositional Semantic Parsing

Although sequence-to-sequence models often achieve good performance in s...
research
05/04/2022

Compositional Task-Oriented Parsing as Abstractive Question Answering

Task-oriented parsing (TOP) aims to convert natural language into machin...
research
10/08/2020

Learning to Recombine and Resample Data for Compositional Generalization

Flexible neural models outperform grammar- and automaton-based counterpa...
research
08/29/2018

Neural Compositional Denotational Semantics for Question Answering

Answering compositional questions requiring multi-step reasoning is chal...
research
04/29/2019

Routing Networks and the Challenges of Modular and Compositional Computation

Compositionality is a key strategy for addressing combinatorial complexi...

Please sign up or login with your details

Forgot password? Click here to reset