Log In Sign Up

Robust and Efficient Sorting with Offset-Value Coding

by   Thanh Do, et al.

Sorting and searching are large parts of database query processing, e.g., in the forms of index creation, index maintenance, and index lookup; and comparing pairs of keys is a substantial part of the effort in sorting and searching. We have worked on simple, efficient implementations of decades-old, neglected, effective techniques for fast comparisons and fast sorting, in particular offset-value coding. In the process, we happened upon its mutually beneficial relationship with prefix truncation in run files as well as the duality of compression techniques in row- and column-format storage structures, namely prefix truncation and run-length encoding of leading key columns. We also found a beneficial relationship with consumers of sorted streams, e.g., merging parallel streams, in-stream aggregation, and merge join. We report on our implementation in the context of Google's Napa and F1 Query systems as well as an experimental evaluation of performance and scalability.


page 1

page 2

page 3

page 4


Offset-value coding in database query processing

Recent work measures how much offset-value coding speeds up database que...

Sort-based grouping and aggregation

Database query processing requires algorithms for duplicate removal, gro...

Space/time-efficient RDF stores based on circular suffix sorting

In recent years, RDF has gained popularity as a format for the standardi...

FLiMS: a Fast Lightweight 2-way Merger for Sorting

In this paper, we present FLiMS, a highly-efficient and simple parallel ...

Compressed Key Sort and Fast Index Reconstruction

In this paper we propose an index key compression scheme based on the no...

Implementing the Comparison-Based External Sort

In the age of big data, sorting is an indispensable operation for DBMSes...

Multiway Powersort

Powersort (Munro Wild, ESA2018) has recently replaced Timsort's subo...