Certifying Certainty and Uncertainty in Approximate Membership Query Structures – Extended Version

04/28/2020
by   Kiran Gopinathan, et al.
0

Approximate Membership Query structures (AMQs) rely on randomisation for time- and space-efficiency, while introducing a possibility of false positive and false negative answers. Correctness proofs of such structures involve subtle reasoning about bounds on probabilities of getting certain outcomes. Because of these subtleties, a number of unsound arguments in such proofs have been made over the years. In this work, we address the challenge of building rigorous and reusable computer-assisted proofs about probabilistic specifications of AMQs. We describe the framework for systematic decomposition of AMQs and their properties into a series of interfaces and reusable components. We implement our framework as a library in the Coq proof assistant and showcase it by encoding in it a number of non-trivial AMQs, such as Bloom filters, counting filters, quotient filters and blocked constructions, and mechanising the proofs of their probabilistic specifications. We demonstrate how AMQs encoded in our framework guarantee the absence of false negatives by construction. We also show how the proofs about probabilities of false positives for complex AMQs can be obtained by means of verified reduction to the implementations of their simpler counterparts. Finally, we provide a library of domain-specific theorems and tactics that allow a high degree of automation in probabilistic proofs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/12/2021

Approximate Membership Query Filters with a False Positive Free Set

In the last decade, significant efforts have been made to reduce the fal...
research
09/04/2021

Stretching Your Data With Taffy Filters

Popular approximate membership query structures such as Bloom filters an...
research
01/18/2018

NAE-SAT-based probabilistic membership filters

Probabilistic membership filters are a type of data structure designed t...
research
11/05/2017

Bloom Filters, Adaptivity, and the Dictionary Problem

The Bloom filter---or, more generally, an approximate membership query d...
research
06/27/2020

Optimizing Cuckoo Filter for high burst tolerance,low latency, and high throughput

In this paper, we present an implementation of a cuckoo filter for membe...
research
08/21/2023

Proofs about Network Communication: For Humans and Machines

Many concurrent and distributed systems are safety-critical and therefor...
research
09/14/2022

Adversarial Correctness and Privacy for Probabilistic Data Structures

We study the security of Probabilistic Data Structures (PDS) for handlin...

Please sign up or login with your details

Forgot password? Click here to reset