A Unified Deep Model of Learning from both Data and Queries for Cardinality Estimation

07/26/2021
by   Peizhi Wu, et al.
0

Cardinality estimation is a fundamental problem in database systems. To capture the rich joint data distributions of a relational table, most of the existing work either uses data as unsupervised information or uses query workload as supervised information. Very little work has been done to use both types of information, and cannot fully make use of both types of information to learn the joint data distribution. In this work, we aim to close the gap between data-driven and query-driven methods by proposing a new unified deep autoregressive model, UAE, that learns the joint data distribution from both the data and query workload. First, to enable using the supervised query information in the deep autoregressive model, we develop differentiable progressive sampling using the Gumbel-Softmax trick. Second, UAE is able to utilize both types of information to learn the joint data distribution in a single model. Comprehensive experimental results demonstrate that UAE achieves single-digit multiplicative error at tail, better accuracies over state-of-the-art methods, and is both space and time efficient.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/02/2019

DeepDB: Learn from Data, not from Queries!

The typical approach for learned DBMS components is to capture the behav...
research
05/10/2019

Selectivity Estimation with Deep Likelihood Models

Selectivity estimation has long been grounded in statistical tools for d...
research
07/25/2023

Duet: efficient and scalable hybriD neUral rElation undersTanding

Learned cardinality estimation methods have achieved high precision comp...
research
03/24/2019

Multi-Attribute Selectivity Estimation Using Deep Learning

Selectivity estimation - the problem of estimating the result size of qu...
research
05/28/2023

One stone, two birds: A lightweight multidimensional learned index with cardinality support

Innovative learning based structures have recently been proposed to tack...
research
01/01/2021

SetSketch: Filling the Gap between MinHash and HyperLogLog

MinHash and HyperLogLog are sketching algorithms that have become indisp...
research
05/06/2014

Is Joint Training Better for Deep Auto-Encoders?

Traditionally, when generative models of data are developed via deep arc...

Please sign up or login with your details

Forgot password? Click here to reset