The Case for Learned In-Memory Joins

by   Ibrahim Sabek, et al.

In-memory join is an essential operator in any database engine. It has been extensively investigated in the database literature. In this paper, we study whether exploiting the CDF-based learned models to boost the join performance is practical or not. To the best of our knowledge, we are the first to fill this gap. We investigate the usage of CDF-based partitioning and learned indexes (e.g., RMI and RadixSpline) in the three join categories; indexed nested loop join(INLJ), sort-based joins (SJ) and hash-based joins (HJ). We proposed new efficient learned variants for the INLJ and SJ categories. In addition, we proposed a reinforcement learning based optimizer to select the best join algorithm, whether learned or not-learned, for each join query. Our experimental analysis showed that our learned joins variants of INLJ and SJ consistently outperform the state-of-the-art techniques.



There are no comments yet.


page 10


Deep Reinforcement Learning for Join Order Enumeration

Join order selection plays a significant role in query performance. Many...

Parallel In-Memory Evaluation of Spatial Joins

The spatial join is a popular operation in spatial database systems and ...

Design Trade-offs for a Robust Dynamic Hybrid Hash Join (Extended Version)

The Join operator, as one of the most expensive and commonly used operat...

Unsupervised Learning through Prediction in a Model of Cortex

We propose a primitive called PJOIN, for "predictive join," which combin...

Checkpointing and Localized Recovery for Nested Fork-Join Programs

While checkpointing is typically combined with a restart of the whole ap...

Efficiently Charting RDF

We propose a visual query language for interactively exploring large-sca...

Hiperfact: In-Memory High Performance Fact Processing – Rethinking the Rete Inference Algorithm

The Rete forward inference algorithm forms the basis for many rule engin...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.