Separations for Estimating Large Frequency Moments on Data Streams
share
We study the classical problem of moment estimation of an underlying vector whose n coordinates are implicitly defined through a series of updates in a data stream. We show that if the updates to the vector arrive in the random-order insertion-only model, then there exist space efficient algorithms with improved dependencies on the approximation parameter ε. In particular, for any real p > 2, we first obtain an algorithm for F_p moment estimation using 𝒪̃(1/ε^4/p· n^1-2/p) bits of memory. Our techniques also give algorithms for F_p moment estimation with p>2 on arbitrary order insertion-only and turnstile streams, using 𝒪̃(1/ε^4/p· n^1-2/p) bits of space and two passes, which is the first optimal multi-pass F_p estimation algorithm up to log n factors. Finally, we give an improved lower bound of Ω(1/ε^2· n^1-2/p) for one-pass insertion-only streams. Our results separate the complexity of this problem both between random and non-random orders, as well as one-pass and multi-pass streams.
READ FULL TEXT