DeepAI
Log In Sign Up

Techniques for Inverted Index Compression

08/28/2019
by   Giulio Ermanno Pibiri, et al.
0

The data structure at the core of large-scale search engines is the inverted index, which is essentially a collection of sorted integer sequences called inverted lists. Because of the many documents indexed by such engines and stringent performance requirements imposed by the heavy load of queries, the inverted index stores billions of integers that must be searched efficiently. In this scenario, index compression is essential because it leads to a better exploitation of the computer memory hierarchy for faster query processing and, at the same time, allows reducing the number of storage machines. The aim of this article is twofold: first, surveying the encoding algorithms suitable for inverted index compression and, second, characterizing the performance of the inverted index through experimentation.

READ FULL TEXT

page 12

page 30

09/24/2020

Compressed Key Sort and Fast Index Reconstruction

In this paper we propose an index key compression scheme based on the no...
07/01/2019

On Slicing Sorted Integer Sequences

Representing sorted integer sequences in small space is a central proble...
12/26/2021

Airphant: Cloud-oriented Document Indexing

Modern data warehouses can scale compute nodes independently of storage....
09/05/2022

Compressing integer lists with Contextual Arithmetic Trits

Inverted indexes allow to query large databases without needing to searc...
06/09/2020

Faster Queries on BWT-runs Compressed Indexes

Although a significant number of compressed indexes for highly repetitiv...
10/07/2019

ER-index: a referential index for encrypted genomic databases

Huge DBMSs storing genomic information are being created and engineerize...
02/11/2019

CPOI: A Compact Method to Archive Versioned RDF Triple-Sets

Large amounts of RDF/S data are produced and published lately, and sever...