Designing Succinct Secondary Indexing Mechanism by Exploiting Column Correlations

03/27/2019
by   Yingjun Wu, et al.
0

Database administrators construct secondary indexes on data tables to accelerate query processing in relational database management systems (RDBMSs). These indexes are built on top of the most frequently queried columns according to the data statistics. Unfortunately, maintaining multiple secondary indexes in the same database can be extremely space consuming, causing significant performance degradation due to the potential exhaustion of memory space. In this paper, we demonstrate that there exist many opportunities to exploit column correlations for accelerating data access. We propose HERMIT, a succinct secondary indexing mechanism for modern RDBMSs. HERMIT judiciously leverages the rich soft functional dependencies hidden among columns to prune out redundant structures for indexed key access. Instead of building a complete index that stores every single entry in the key columns, HERMIT navigates any incoming key access queries to an existing index built on the correlated columns. This is achieved through the Tiered Regression Search Tree (TRS-Tree), a succinct, ML-enhanced data structure that performs fast curve fitting to adaptively and dynamically capture both column correlations and outliers. Our extensive experimental study in two different RDBMSs have confirmed that HERMIT can significantly reduce space consumption with limited performance overhead, especially when supporting complex range queries.

READ FULL TEXT
research
03/27/2019

Designing Succinct Secondary Indexing Mechanism by Exploiting Column Correlations (Extended Version)

Database administrators construct secondary indexes on data tables to ac...
research
12/12/2020

Cortex: Harnessing Correlations to Boost Query Performance

Databases employ indexes to filter out irrelevant records, which reduces...
research
06/29/2020

Leveraging Soft Functional Dependencies for Indexing Multi-dimensional Data

A new proposal in database indexing has been for index structures to aut...
research
05/11/2022

LSI: A Learned Secondary Index Structure

Learned index structures have been shown to achieve favorable lookup per...
research
09/29/2021

Relational Memory: Native In-Memory Accesses on Rows and Columns

Analytical database systems are typically designed to use a column-first...
research
10/31/2017

Extracting Syntactic Patterns from Databases

Many database columns contain string or numerical data that conforms to ...

Please sign up or login with your details

Forgot password? Click here to reset