PanJoin: A Partition-based Adaptive Stream Join

11/13/2018
by   Fei Pan, et al.
0

In stream processing, stream join is one of the critical sources of performance bottlenecks. The sliding-window-based stream join provides a precise result but consumes considerable computational resources. The current solutions lack support for the join predicates on large windows. These algorithms and their hardware accelerators are either limited to equi-join or use a nested loop join to process all the requests. In this paper, we present a new algorithm called PanJoin which has high throughput on large windows and supports both equi-join and non-equi-join. PanJoin implements three new data structures to reduce computations during the probing phase of stream join. We also implement the most hardware-friendly data structure, called BI-Sort, on FPGA. Our evaluation shows that PanJoin outperforms several recently proposed stream join methods by more than 1000x, and it also adapts well to highly skewed data.

READ FULL TEXT
research
10/15/2019

Optimizing Semi-Stream CACHEJOIN for Near-Real-Time Data Warehousing

Streaming data join is a critical process in the field of near-real-time...
research
03/01/2019

Parallel Index-based Stream Join on a Multicore CPU

There is increasing interest in using multicore processors to accelerate...
research
06/21/2022

Graphical Join: A New Physical Join Algorithm for RDBMSs

Join operations (especially n-way, many-to-many joins) are known to be t...
research
03/22/2022

Non-recursive Approach for Sort-Merge Join Operation

Several algorithms have been developed over the years to perform join op...
research
08/05/2022

Towards Fast Theta-join: A Prefiltering and Amalgamated Partitioning Approach

As one of the most useful online processing techniques, the theta-join o...
research
05/17/2019

The TrieJax Architecture: Accelerating Graph Operations Through Relational Joins

Graph pattern matching (e.g., finding all cycles and cliques) has become...
research
07/27/2017

Approximations and Bounds for (n, k) Fork-Join Queues: A Linear Transformation Approach

Compared to basic fork-join queues, a job in (n, k) fork-join queues onl...

Please sign up or login with your details

Forgot password? Click here to reset