Better Sum Estimation via Weighted Sampling

10/28/2021
by   Lorenzo Beretta, et al.
0

Given a large set U where each item a∈ U has weight w(a), we want to estimate the total weight W=∑_a∈ U w(a) to within factor of 1±ε with some constant probability >1/2. Since n=|U| is large, we want to do this without looking at the entire set U. In the traditional setting in which we are allowed to sample elements from U uniformly, sampling Ω(n) items is necessary to provide any non-trivial guarantee on the estimate. Therefore, we investigate this problem in different settings: in the proportional setting we can sample items with probabilities proportional to their weights, and in the hybrid setting we can sample both proportionally and uniformly. These settings have applications, for example, in sublinear-time algorithms and distribution testing. Sum estimation in the proportional and hybrid setting has been considered before by Motwani, Panigrahy, and Xu [ICALP, 2007]. In their paper, they give both upper and lower bounds in terms of n. Their bounds are near-matching in terms of n, but not in terms of ε. In this paper, we improve both their upper and lower bounds. Our bounds are matching up to constant factors in both settings, in terms of both n and ε. No lower bounds with dependency on ε were known previously. In the proportional setting, we improve their Õ(√(n)/ε^7/2) algorithm to O(√(n)/ε). In the hybrid setting, we improve Õ(√(n)/ ε^9/2) to O(√(n)/ε^4/3). Our algorithms are also significantly simpler and do not have large constant factors. We also investigate the previously unexplored setting where n is unknown to the algorithm. Finally, we show how our techniques apply to the problem of edge counting in graphs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/21/2023

On the Optimal Bounds for Noisy Computing

We revisit the problem of computing with noisy information considered in...
research
08/15/2023

Improved Lower Bound for Estimating the Number of Defective Items

Let X be a set of items of size n that contains some defective items, de...
research
04/23/2020

Non-Adaptive Adaptive Sampling on Turnstile Streams

Adaptive sampling is a useful algorithmic tool for data summarization pr...
research
10/24/2020

On Testing of Samplers

Given a set of items ℱ and a weight function 𝚠𝚝: ℱ↦ (0,1), the problem o...
research
10/03/2022

Online Pen Testing

We study a "pen testing" problem, in which we are given n pens with unkn...
research
01/27/2018

Approximate Inference via Weighted Rademacher Complexity

Rademacher complexity is often used to characterize the learnability of ...
research
05/30/2023

Optimal Dynamic Subset Sampling: Theory and Applications

We study the fundamental problem of sampling independent events, called ...

Please sign up or login with your details

Forgot password? Click here to reset