String Synchronizing Sets: Sublinear-Time BWT Construction and Optimal LCE Data Structure
share
Burrows-Wheeler transform (BWT) is an invertible text transformation that, given a text T of length n, permutes its symbols according to the lexicographic order of suffixes of T. BWT is one of the most heavily studied algorithms in data compression with numerous applications in indexing, sequence analysis, and bioinformatics. Its construction is a bottleneck in many scenarios and settling the complexity of this task is one of the most important unsolved problems in sequence analysis that has remained open for 25 years. Given a binary string of length n occupying O(n/ n) machine words of space, the BWT construction algorithm due to Hon et al. (FOCS 2003) runs in O(n) time and O(n/ n) space. Recent advancements (Belazzougui, STOC 2014, and Munro et al., SODA 2017) focus on removing the alphabet-size dependency in the time complexity, but they still require Ω(n) time. In this paper, we propose the first algorithm that breaks the O(n)-time barrier for BWT construction. Given a binary string of length n, our procedure builds the Burrows-Wheeler transform in O(n/√( n)) time and O(n/ n) space. We complement this result with a conditional lower bound proving that any further progress in the time complexity of BWT construction would yield faster algorithms for the very well studied problem of counting inversions: it would improve the state-of-the-art O(m√( m))-time solution by Chan and Pǎtraşcu (SODA 2010). Our algorithm is based on a novel concept of string synchronizing sets. As one of the applications, we show a data structure of the optimal size O(n/ n) that answers longest common extension queries in O(1) time and, furthermore, can be deterministically constructed in the optimal O(n/ n) time. This significantly improves upon the previously best data structures and essentially closes the LCE problem.
READ FULL TEXT