On the Structural Generalization in Text-to-SQL

01/12/2023
by   Jieyu Li, et al.
0

Exploring the generalization of a text-to-SQL parser is essential for a system to automatically adapt the real-world databases. Previous works provided investigations focusing on lexical diversity, including the influence of the synonym and perturbations in both natural language questions and databases. However, research on the structure variety of database schema (DS) is deficient. Specifically, confronted with the same input question, the target SQL is probably represented in different ways when the DS comes to a different structure. In this work, we provide in-deep discussions about the structural generalization of text-to-SQL tasks. We observe that current datasets are too templated to study structural generalization. To collect eligible test data, we propose a framework to generate novel text-to-SQL data via automatic and synchronous (DS, SQL) pair altering. In the experiments, significant performance reduction when evaluating well-trained text-to-SQL models on the synthetic samples demonstrates the limitation of current research regarding structural generalization. According to comprehensive analysis, we suggest the practical reason is the overfitting of (NL, SQL) patterns.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/21/2023

Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness

Neural text-to-SQL models have achieved remarkable performance in transl...
research
10/21/2020

DuoRAT: Towards Simpler Text-to-SQL Models

Recent research has shown that neural text-to-SQL models can effectively...
research
05/04/2022

Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment

In text-to-SQL tasks – as in much of NLP – compositional generalization ...
research
10/23/2022

Towards Generalizable and Robust Text-to-SQL Parsing

Text-to-SQL parsing tackles the problem of mapping natural language ques...
research
10/29/2022

Diverse Parallel Data Synthesis for Cross-Database Adaptation of Text-to-SQL Parsers

Text-to-SQL parsers typically struggle with databases unseen during the ...
research
05/29/2020

ValueNet: A Neural Text-to-SQL Architecture Incorporating Values

Building natural language interfaces for databases has been a long-stand...
research
10/20/2019

Byte-Pair Encoding for Text-to-SQL Generation

Neural sequence-to-sequence models provide a competitive approach to the...

Please sign up or login with your details

Forgot password? Click here to reset