Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures

07/17/2020
by   Daniel Furrer, et al.
0

While mainstream machine learning methods are known to have limited ability to compositionally generalize, new architectures and techniques continue to be proposed to address this limitation. We investigate state-of-the-art techniques and architectures in order to assess their effectiveness in improving compositional generalization in semantic parsing tasks based on the SCAN and CFQ datasets. We show that masked language model (MLM) pre-training rivals SCAN-inspired architectures on primitive holdout splits. On a more complex compositional task, we show that pre-training leads to significant improvements in performance vs. comparable non-pre-trained models, whereas architectures proposed to encourage compositional generalization on SCAN or in the area of algorithm learning fail to lead to significant improvements. We establish a new state of the art on the CFQ compositional generalization benchmark using MLM pre-training together with an intermediate representation.

READ FULL TEXT
research
10/24/2020

Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?

Sequence-to-sequence models excel at handling natural language variation...
research
09/29/2022

Compositional Semantic Parsing with Large Language Models

Humans can reason compositionally when presented with new tasks. Previou...
research
11/09/2021

Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks

Neural network models often generalize poorly to mismatched domains or d...
research
10/07/2019

Compositional Generalization for Primitive Substitutions

Compositional generalization is a basic mechanism in human language lear...
research
11/14/2019

A Scalable Approach for Facial Action Unit Classifier Training UsingNoisy Data for Pre-Training

Machine learning systems are being used to automate many types of labori...
research
12/15/2020

*-CFQ: Analyzing the Scalability of Machine Learning on a Compositional Task

We present *-CFQ ("star-CFQ"): a suite of large-scale datasets of varyin...
research
12/20/2019

Measuring Compositional Generalization: A Comprehensive Method on Realistic Data

State-of-the-art machine learning methods exhibit limited compositional ...

Please sign up or login with your details

Forgot password? Click here to reset