# Linear Probing Revisited: Tombstones Mark the Death of Primary Clustering

First introduced in 1954, linear probing is one of the oldest data structures in computer science, and due to its unrivaled data locality, it continues to be one of the fastest hash tables in practice. It is widely believed and taught, however, that linear probing should never be used at high load factors; this is because primary-clustering effects cause insertions at load factor 1 - 1 /x to take expected time Θ(x^2) (rather than the ideal Θ(x)). The dangers of primary clustering, first discovered by Knuth in 1963, have been taught to generations of computer scientists, and have influenced the design of some of many widely used hash tables. We show that primary clustering is not a foregone conclusion. We demonstrate that small design decisions in how deletions are implemented have dramatic effects on the asymptotic performance of insertions, so that, even if a hash table operates continuously at a load factor 1 - Θ(1/x), the expected amortized cost per operation is Õ(x). This is because tombstones created by deletions actually cause an anti-clustering effect that combats primary clustering. We also present a new variant of linear probing (which we call graveyard hashing) that completely eliminates primary clustering on any sequence of operations: if, when an operation is performed, the current load factor is 1 - 1/x for some x, then the expected cost of the operation is O(x). One corollary is that, in the external-memory model with a data blocks of size B, graveyard hashing offers the following remarkable guarantee: at any load factor 1 - 1/x satisfying x = o(B), graveyard hashing achieves 1 + o(1) expected block transfers per operation. Past external-memory hash tables have only been able to offer a 1 + o(1) guarantee when the block size B is at least Ω(x^2).

## Authors

• 12 publications
• 2 publications
• 21 publications
• ### DHash: Enabling Dynamic and Efficient Hash Tables

Given a specified average load factor, hash tables offer the appeal of c...
06/01/2020 ∙ by Junchang Wang, et al. ∙ 0

• ### Dash: Scalable Hashing on Persistent Memory

Byte-addressable persistent memory (PM) brings hash tables the potential...
03/16/2020 ∙ by Baotong Lu, et al. ∙ 0

• ### All-Purpose Hashing

Despite being one of the oldest data structures in computer science, has...
09/09/2021 ∙ by Michael A. Bender, et al. ∙ 0

• ### WarpCore: A Library for fast Hash Tables on GPUs

Hash tables are ubiquitous. Properties such as an amortized constant tim...
09/16/2020 ∙ by Daniel Jünger, et al. ∙ 0

• ### Data-Parallel Hashing Techniques for GPU Architectures

Hash tables are one of the most fundamental data structures for effectiv...
07/11/2018 ∙ by Brenton Lessley, et al. ∙ 0

• ### A Genetic Algorithm for Obtaining Memory Constrained Near-Perfect Hashing

The problem of fast items retrieval from a fixed collection is often enc...
07/16/2020 ∙ by Dan Domnita, et al. ∙ 0