HashGraph – Scalable Hash Tables Using A Sparse Graph Data Structure

by   Oded Green, et al.

Hash tables are ubiquitous and used in a wide range of applications for efficient probing of large and unsorted data. If designed properly, hash-tables can enable efficients look ups in a constant number of operations or commonly referred to as O(1) operations. As data sizes continue to grow and data becomes less structured (as is common for big-data applications), the need for efficient and scalable hash table also grows. In this paper we introduce HashGraph, a new scalable approach for building hash tables that uses concepts taken from sparse graph representations–hence the name HashGraph. We show two different variants of HashGraph, a simple algorithm that outlines the method to create the hash-table and an advanced method that creates the hash table in a more efficient manner (with an improved memory access pattern). HashGraph shows a new way to deal with hash-collisions that does not use "open-addressing" or "chaining", yet has all the benefits of both these approaches. HashGraph currently works for static inputs, though recent progress with dynamic graph data structures suggest that HashGraph might be extended to dynamic inputs as well. We show that HashGraph can deal with a large number of hash-values per entry without loss of performance as most open-addressing and chaining approaches have. Further, we show that HashGraph is indifferent to the load-factor. Lastly, we show a new probing algorithm for the second phase of value lookups. Given the above, HashGraph is extremely fast and outperforms several state of the art hash-table implementations. The implementation of HashGraph in this paper is for NVIDIA GPUs, though HashGraph is not architecture dependent. Using a NVIDIA GV100 GPU, HashGraph is anywhere from 2X-8X faster than cuDPP, WarpDrive, and cuDF. HashGraph is able to build a hash-table at a rate of 2.5 billion keys per second and can probe at nearly the same rate.


page 1

page 2

page 3

page 4


Better GPU Hash Tables

We revisit the problem of building static hash tables on the GPU and des...

WarpCore: A Library for fast Hash Tables on GPUs

Hash tables are ubiquitous. Properties such as an amortized constant tim...

Wormhole: A Fast Ordered Index for In-memory Data Management

In-memory data management systems, such as key-value store, have become ...

DHash: Enabling Dynamic and Efficient Hash Tables

Given a specified average load factor, hash tables offer the appeal of c...

Defending Hash Tables from Subterfuge with Depth Charge

We consider the problem of defending a hash table against a Byzantine at...

Quotient Hash Tables - Efficiently Detecting Duplicates in Streaming Data

This article presents the Quotient Hash Table (QHT) a new data structure...

When Are Learned Models Better Than Hash Functions?

In this work, we aim to study when learned models are better hash functi...

Please sign up or login with your details

Forgot password? Click here to reset