String Synchronizing Sets: Sublinear-Time BWT Construction and Optimal LCE Data Structure

by   Dominik Kempa, et al.

Burrows-Wheeler transform (BWT) is an invertible text transformation that, given a text T of length n, permutes its symbols according to the lexicographic order of suffixes of T. BWT is one of the most heavily studied algorithms in data compression with numerous applications in indexing, sequence analysis, and bioinformatics. Its construction is a bottleneck in many scenarios and settling the complexity of this task is one of the most important unsolved problems in sequence analysis that has remained open for 25 years. Given a binary string of length n occupying O(n/ n) machine words of space, the BWT construction algorithm due to Hon et al. (FOCS 2003) runs in O(n) time and O(n/ n) space. Recent advancements (Belazzougui, STOC 2014, and Munro et al., SODA 2017) focus on removing the alphabet-size dependency in the time complexity, but they still require Ω(n) time. In this paper, we propose the first algorithm that breaks the O(n)-time barrier for BWT construction. Given a binary string of length n, our procedure builds the Burrows-Wheeler transform in O(n/√( n)) time and O(n/ n) space. We complement this result with a conditional lower bound proving that any further progress in the time complexity of BWT construction would yield faster algorithms for the very well studied problem of counting inversions: it would improve the state-of-the-art O(m√( m))-time solution by Chan and Pǎtraşcu (SODA 2010). Our algorithm is based on a novel concept of string synchronizing sets. As one of the applications, we show a data structure of the optimal size O(n/ n) that answers longest common extension queries in O(1) time and, furthermore, can be deterministically constructed in the optimal O(n/ n) time. This significantly improves upon the previously best data structures and essentially closes the LCE problem.


page 1

page 2

page 3

page 4


Closing in on Time and Space Optimal Construction of Compressed Indexes

Fast and space-efficient construction of compressed indexes such as comp...

Locally Consistent Parsing for Text Indexing in Small Space

We consider two closely related problems of text indexing in a sub-linea...

Translating Between Wavelet Tree and Wavelet Matrix Construction

The wavelet tree (Grossi et al. [SODA, 2003]) and wavelet matrix (Claude...

A New Algebraic Approach for String Reconstruction from Substring Compositions

We consider the problem of binary string reconstruction from the multise...

Substring Complexity in Sublinear Space

Shannon's entropy is a definitive lower bound for statistical compressio...

Optimal Substring-Equality Queries with Applications to Sparse Text Indexing

We consider the problem of encoding a string of length n from an alphabe...

The Alternating BWT: an algorithmic perspective

The Burrows-Wheeler Transform (BWT) is a word transformation introduced ...

Please sign up or login with your details

Forgot password? Click here to reset