DeepAI AI Chat
Log In Sign Up

Compositional pre-training for neural semantic parsing

by   Amir Ziai, et al.
Stanford University

Semantic parsing is the process of translating natural language utterances into logical forms, which has many important applications such as question answering and instruction following. Sequence-to-sequence models have been very successful across many NLP tasks. However, a lack of task-specific prior knowledge can be detrimental to the performance of these models. Prior work has used frameworks for inducing grammars over the training examples, which capture conditional independence properties that the model can leverage. Inspired by the recent success stories such as BERT we set out to extend this augmentation framework into two stages. The first stage is to pre-train using a corpus of augmented examples in an unsupervised manner. The second stage is to fine-tune to a domain-specific task. In addition, since the pre-training stage is separate from the training on the main task we also expand the universe of possible augmentations without causing catastrophic inference. We also propose a novel data augmentation strategy that interchanges tokens that co-occur in similar contexts to produce new training pairs. We demonstrate that the proposed two-stage framework is beneficial for improving the parsing accuracy in a standard dataset called GeoQuery for the task of generating logical forms from a set of questions about the US geography.


page 1

page 2

page 3

page 4


TAPAS: Weakly Supervised Table Parsing via Pre-training

Answering natural language questions over tables is usually seen as a se...

Data Recombination for Neural Semantic Parsing

Modeling crisp logical regularities is crucial in semantic parsing, maki...

Improving Compositional Generalization with Latent Structure and Data Augmentation

Generic unstructured neural networks have been shown to struggle on out-...

Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing

One daunting problem for semantic parsing is the scarcity of annotation....

Generating Logical Forms from Graph Representations of Text and Entities

Structured information about entities is critical for many semantic pars...

Compositionality as Lexical Symmetry

Standard deep network models lack the inductive biases needed to general...

On Robustness of Neural Semantic Parsers

Semantic parsing maps natural language (NL) utterances into logical form...