Understanding the hardness of approximate query processing with joins

10/01/2020
by   Tianyu Liu, et al.
0

We study the hardness of Approximate Query Processing (AQP) of various types of queries involving joins over multiple tables of possibly different sizes. In the case where the query result is a single value (e.g., COUNT, SUM, and COUNT(DISTINCT)), we prove worst-case information-theoretic lower bounds for AQP problems that are given parameters ϵ and δ, and return estimated results within a factor of 1+ϵ of the true results with error probability at most δ. In particular, the lower bounds for cardinality estimation over joins under various settings are contained in our results. Informally, our results show that for various database queries with joins, unless restricted to the set of queries whose results are always guaranteed to be above a very large threshold, the amount of information an AQP algorithm needs for returning an accurate approximation is at least linear in the number of rows in the largest table. Similar lower bounds even hold for some special cases where additional information such as top-K heavy hitters and all frequency vectors are available. In the case of GROUP-BY where the query result is not a single number, we study the lower bound for the amount of information used by any approximation algorithm that does not report any non-existing group and does not miss groups of large total size. Our work extends the work of Alon, Gibbons, Matias, and Szegedy [AGMS99].We compare our lower bounds with the amount of information required by Bernoulli sampling to give an accurate approximation. For COUNT queries with joins over multiple tables of the same size, the upper bound matches the lower bound, unless the problem setting is restricted to the set of queries whose results are always guaranteed to be above a very large threshold.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/02/2019

On the Hardness of Set Disjointness and Set Intersection with Bounded Universe

In the SetDisjointness problem, a collection of m sets S_1,S_2,...,S_m f...
research
02/10/2019

Set Cover in Sub-linear Time

We study the classic set cover problem from the perspective of sub-linea...
research
12/19/2021

Conditional Lower Bounds for Dynamic Geometric Measure Problems

We give new polynomial lower bounds for a number of dynamic measure prob...
research
04/04/2022

An Optimal Algorithm for Certifying Monotone Functions

Given query access to a monotone function f{0,1}^n→{0,1} with certificat...
research
02/03/2022

Answering Count Queries for Genomic Data with Perfect Privacy

In this paper, we consider the problem of answering count queries for ge...
research
12/02/2021

Generalized Framework for Group Testing: Queries, Feedbacks and Adversaries

In the Group Testing problem, the objective is to learn a subset K of so...
research
08/21/2018

Group Activity Selection with Few Agent Types

The Group Activity Selection Problem (GASP) models situations where a gr...

Please sign up or login with your details

Forgot password? Click here to reset