Fast Partitioned Learned Bloom Filter

06/05/2023
by   Atsuki Sato, et al.
0

A Bloom filter is a memory-efficient data structure for approximate membership queries used in numerous fields of computer science. Recently, learned Bloom filters that achieve better memory efficiency using machine learning models have attracted attention. One such filter, the partitioned learned Bloom filter (PLBF), achieves excellent memory efficiency. However, PLBF requires a O(N^3k) time complexity to construct the data structure, where N and k are the hyperparameters of PLBF. One can improve memory efficiency by increasing N, but the construction time becomes extremely long. Thus, we propose two methods that can reduce the construction time while maintaining the memory efficiency of PLBF. First, we propose fast PLBF, which can construct the same data structure as PLBF with a smaller time complexity O(N^2k). Second, we propose fast PLBF++, which can construct the data structure with even smaller time complexity O(Nklog N + Nk^2). Fast PLBF++ does not necessarily construct the same data structure as PLBF. Still, it is almost as memory efficient as PLBF, and it is proved that fast PLBF++ has the same data structure as PLBF when the distribution satisfies a certain constraint. Our experimental results from real-world datasets show that (i) fast PLBF and fast PLBF++ can construct the data structure up to 233 and 761 times faster than PLBF, (ii) fast PLBF can achieve the same memory efficiency as PLBF, and (iii) fast PLBF++ can achieve almost the same memory efficiency as PLBF.

READ FULL TEXT

page 7

page 22

page 24

research
03/17/2019

Shed More Light on Bloom Filter's Variants

Bloom Filter is a probabilistic membership data structure and it is exce...
research
03/15/2019

scaleBF: A High Scalable Membership Filter using 3D Bloom Filter

Bloom Filter is extensively deployed data structure in various applicati...
research
12/17/2019

Xor Filters: Faster and Smaller Than Bloom and Cuckoo Filters

The Bloom filter provides fast approximate set membership while using li...
research
11/05/2019

An Efficient Word Lookup System by using Improved Trie Algorithm

Efficiently word storing and searching is an important task in computer ...
research
01/17/2020

Circllhist – A Log-Linear Histogram Data Structure for IT Infrastructure Monitoring

The circllhist histogram is a fast and memory efficient data structure f...
research
09/04/2021

Fast Succinct Retrieval and Approximate Membership using Ribbon

A retrieval data structure for a static function f:S→{0,1}^r supports qu...
research
10/15/2018

Preventing DDoS using Bloom Filter: A Survey

Distributed Denial-of-Service (DDoS) is a menace for service provider an...

Please sign up or login with your details

Forgot password? Click here to reset