DeepAI
Log In Sign Up

Compressed Key Sort and Fast Index Reconstruction

09/24/2020
by   Yongsik Kwon, et al.
0

In this paper we propose an index key compression scheme based on the notion of distinction bits by proving that the distinction bits of index keys are sufficient information to determine the sorted order of the index keys correctly. While the actual compression ratio may vary depending on the characteristics of datasets (an average of 2.76 to one compression ratio was observed in our experiments), the index key compression scheme leads to significant performance improvements during the reconstruction of large-scale indexes. Our index key compression can be effectively used in database replication and index recovery of modern main-memory database systems.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/28/2019

Techniques for Inverted Index Compression

The data structure at the core of large-scale search engines is the inve...
12/08/2021

FastSGD: A Fast Compressed SGD Framework for Distributed Machine Learning

With the rapid increase of big data, distributed Machine Learning (ML) h...
05/24/2022

NFL: Robust Learned Index via Distribution Transformation

Recent works on learned index open a new direction for the indexing fiel...
05/11/2017

ROCKER: A Refinement Operator for Key Discovery

The Linked Data principles provide a decentral approach for publishing s...
09/16/2021

SEACOW: Synopsis Embedded Array Compression using Wavelet Transform

Recently, multidimensional data is produced in various domains; because ...
09/17/2022

Robust and Efficient Sorting with Offset-Value Coding

Sorting and searching are large parts of database query processing, e.g....
06/22/2022

ROIBIN-SZ: Fast and Science-Preserving Compression for Serial Crystallography

Crystallography is the leading technique to study atomic structures of p...