Log In Sign Up

Compressed Key Sort and Fast Index Reconstruction

by   Yongsik Kwon, et al.

In this paper we propose an index key compression scheme based on the notion of distinction bits by proving that the distinction bits of index keys are sufficient information to determine the sorted order of the index keys correctly. While the actual compression ratio may vary depending on the characteristics of datasets (an average of 2.76 to one compression ratio was observed in our experiments), the index key compression scheme leads to significant performance improvements during the reconstruction of large-scale indexes. Our index key compression can be effectively used in database replication and index recovery of modern main-memory database systems.


page 1

page 2

page 3

page 4


Techniques for Inverted Index Compression

The data structure at the core of large-scale search engines is the inve...

FastSGD: A Fast Compressed SGD Framework for Distributed Machine Learning

With the rapid increase of big data, distributed Machine Learning (ML) h...

NFL: Robust Learned Index via Distribution Transformation

Recent works on learned index open a new direction for the indexing fiel...

ROCKER: A Refinement Operator for Key Discovery

The Linked Data principles provide a decentral approach for publishing s...

SEACOW: Synopsis Embedded Array Compression using Wavelet Transform

Recently, multidimensional data is produced in various domains; because ...

Robust and Efficient Sorting with Offset-Value Coding

Sorting and searching are large parts of database query processing, e.g....

ROIBIN-SZ: Fast and Science-Preserving Compression for Serial Crystallography

Crystallography is the leading technique to study atomic structures of p...