ExtendedHyperLogLog: Analysis of a new Cardinality Estimator

06/11/2021
by   Tal Ohayon, et al.
0

We discuss the problem of counting distinct elements in a stream. A stream is usually considered as a sequence of elements that come one at a time. An exact solution to the problem requires memory space of the size of the stream. For many applications this solution is infeasible due to very large streams. The solution in that case, is to use a probabilistic data structure (also called sketch), from which we can estimate with high accuracy the cardinality of the stream. We present a new algorithm, ExtendedHyperLogLog (EHLL), which is based on the state-of-the-art algorithm, HyperLogLog (HLL). In order to achieve the same accuracy as HLL, EHLL uses 16 approach has bean developed. In the martingale setting we receive better accuracy at the price of not being able to merge sketches. EHLL also works in the martingale setting. Martingale EHLL achieves the same accuracy as Martingale HLL using 12

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/03/2023

An embarrassingly parallel optimal-space cardinality estimation algorithm

In 2020 Blasiok (ACM Trans. Algorithms 16(2) 3:1-3:28) constructed an op...
research
03/13/2019

Cardinality Estimation in a Virtualized Network Device Using Online Machine Learning

Cardinality estimation algorithms receive a stream of elements, with pos...
research
01/01/2021

SetSketch: Filling the Gap between MinHash and HyperLogLog

MinHash and HyperLogLog are sketching algorithms that have become indisp...
research
06/23/2022

Taking snapshots from a stream

This work is devoted to a certain class of probabilistic snapshots for e...
research
08/20/2015

The ABACOC Algorithm: a Novel Approach for Nonparametric Classification of Data Streams

Stream mining poses unique challenges to machine learning: predictive mo...
research
10/23/2017

HyperMinHash: MinHash in LogLog space

In this extended abstract, we describe and analyse a streaming probabili...
research
01/24/2023

Distinct Elements in Streams: An Algorithm for the (Text) Book

Given a data stream 𝒟 = ⟨ a_1, a_2, …, a_m ⟩ of m elements where each a_...

Please sign up or login with your details

Forgot password? Click here to reset