In-Place Bijective Burrows-Wheeler Transforms

1 Introduction

The Burrows-Wheeler transform (BWT) [6] is one of the most favored options both for (a) compressing and (b) indexing data sets. On the one hand, compression programs like bzip2 apply the BWT to achieve high compression rates. For that, they leverage the effect that the BWT built on repetitive data tends to have long character runs, which can be compressed by run-length compression, i.e., representing a substring of $ℓ$ a’s by the tuple $(a, ℓ)$ . On the other hand, self-indexing data structures like the FM-index [11] enhance the BWT to a full-text self-index. A combined approach of both compression and indexing is the run-length compressed FM-index [21], representing a BWT with $r_{B W T}$ character runs, i.e., maximal repetitions of a character, run-length compressed in $O (r_{B W T} lg n)$ bits. This representation can be computed directly in run-length compressed space thanks to Policriti and Prezza [30]. The BWT and its run-length compressed representation have been intensively studied during the past decades (e.g., [12, 1, 14] and the references therein). Contrary to that, a variant, called the bijective BWT (BBWT) [16], is far from being well-studied despite its mathematically appealing characteristics¹¹1The BBWT is a bijection between strings without the need of an artificial delimiter needed, e.g., to invert the BWT.. As a matter of fact, we are only aware of one index data structure based on the BBWT [3] and of two non-trivial construction algorithms [5, 2] of the (uncompressed) BBWT, both with the need of additional data structures.

In this article, we shed more light on the connection between the BWT and the BBWT by quadratic time in-place conversion algorithms in Sect. 5 constructing the BWT from the BBWT, or vice versa. We can also perform these conversions in the run-length compressed setting in $O (n lg r / lg lg r)$ time with space linear to the number of the character runs (cf. Sect. 4 and Sect. 4), where $r$ is the sum of character runs in the BWT and the BBWT.

2 Related Work

Given a text $T$ of length $n$ , the BWT of $T$ is the string obtained by assigning $B W T [i]$ to the character preceding the $i$ -th lexicographically smallest suffix of $T$ (or the last character of $T$ if this suffix is the text itself). By this definition, we can construct the BWT with any suffix array [22] construction algorithm. However, storing the suffix array inherently needs $n lg n$ bits of space. Crochemore et al. [9] tackled this space problem with an in-place algorithm constructing the BWT in $O (n^{2})$ online on the reversed text by simulating queries on a dynamic wavelet tree [17] that would be built on the (growing) BWT. They also gave an algorithm for restoring the text in-place in $O (n^{2 + ϵ})$ time.

In the run-length compressed setting, Policriti and Prezza [30] can compute the run-length compressed BWT having $r_{B W T}$ character runs in $O (n lg r_{B W T})$ time while using $O (r_{B W T} lg n)$ bits of space. They additionally presented an adaption of the wavelet tree on run-length compressed texts, yielding a representation using $O (r_{B W T} lg n)$ bits of space with $O (lg r_{B W T})$ query and update time. Finally, practical improvements of the run-length compressed BWT construction were considered by Ohno et al. [29].

The BBWT is the string obtained by assigning $B B W T [i]$ to the last character of the $i$ -th smallest string in the list of all conjugates of the factors of the Lyndon factorization sorted with respect to the $≺_{ω}$ order [23, Def. 4]. Bannai et al. [2] recently revealed a connection between the bijective BWT and suffix sorting by presenting an $O (n)$ time BBWT construction algorithm based on SAIS [28]. With dynamic data structures like a dynamic wavelet tree [27], Bonomo et al. [5] could devise an algorithm computing the BBWT in $O (n lg n / lg lg n)$ time. With nearly the same techniques, Mantaci et al. [24] presented an algorithm computing the BWT (and simultaneously the suffix array if needed) from the Lyndon factorization. All these construction algorithms need however data structures taking $O (n lg n)$ bits of space. However, the latter two (i.e., [5] and [24]) can work in-place by simulating the $L F$ mapping (cf. Sects. 3.5 and 3.4), which we focus on in Sect. 5.1.

3 Preliminaries

Our computational model is the word RAM model with word size $\mathupΩ(lgn)$ . Accessing a word costs $O (1)$ time. An algorithm is called in-place if it uses, besides a rewriteable input, only $O (lg n)$ bits of working space. We write $[{b} (I) . . {e} (I)] = I$ for an interval $I$ of natural numbers.

3.1 Strings

Let $Σ$ denote an integer alphabet of size $σ$ with $σ = n^{O (1)}$ . We call an element $T \in Σ^{*}$ a string. Its length is denoted by $| T |$ . Given an integer $j \in [1 . . | T |]$ , we access the $j$ -th character of $T$ with $T [j]$ . Concatenating a string $T \in Σ^{*}$ $k$ times is abbreviated by $T^{k}$ . A string $T$ is called primitive if there is no string $S \in Σ^{+}$ with $T = S^{k}$ for an integer $k$ with $k \geq 2$ .

When $T$ is represented by the concatenation of $X, Y, Z \in Σ^{*}$ , i.e., $T = XYZ$ , then $X$ , $Y$ and $Z$ are called a prefix, substring and suffix of $T$ , respectively; the prefix $X$ , substring $Y$ , or suffix $Z$ is called proper if $X \neq T$ , $Y \neq T$ , or $Z \neq T$ , respectively. For two integers $i, j$ with $1 \leq i \leq j \leq | T |$ , let $T [i . . j]$ denote the substring of $T$ that begins at position $i$ and ends at position $j$ in $T$ . If $i > j$ , then $T [i . . j]$ is the empty string. In particular, the suffix starting at position $j$ of $T$ is called the $j$ -th suffix of $T$ , and denoted with $T [j . .]$ . An occurrence of a substring $S$ in $T$ is treated as a sub-interval of $[1 . . | T |]$ such that $S = T [{b} (S) . . {e} (S)]$ . The longest common prefix (LCP) of two strings $S$ and $T$ is the longest string that is a prefix of both $S$ and $T$ .

Orders on Strings.

We denote the lexicographic order with $≺_{l e x}$ . Given two strings $S$ and $T$ , $S ≺_{l e x} T$ if $S$ is a prefix of $T$ or there exists an integer $ℓ$ with $1 \leq ℓ \leq min (| S |, | T |)$ such that $S [1 . . ℓ - 1] = T [1 . . ℓ - 1]$ and $S [ℓ] < T [ℓ]$ . Next we define the $≺_{ω}$ order of strings, which is based on the lexicographic order of infinite strings: We write $S ≺_{ω} T$ if the infinite concatenation $S^{ω} := S S S \dots$ is lexicographically smaller than $T^{ω} := T T T \dots$ . For instance, $ab ≺_{l e x} aba$ but $aba ≺_{ω} ab$ .

Rank and Select Queries.

Given a string $T \in Σ^{*}$ , a character $c \in Σ$ , and an integer $j$ , the rank query $T . {rank}_{c} (j)$ counts the occurrences of c in $T [1 . . j]$ , and the select query $T . {select}_{c} (j)$ gives the position of the $j$ -th c in $T$ . We stipulate that ${rank}_{c} (0) = {select}_{c} (0) = 0$ . A wavelet tree is a data structure supporting rank and select queries.

3.2 Lyndon Words

Given a string $T = T [1 . . n]$ , its $i$ -th conjugate ${conj}_{i} (T)$ is defined as $T [i + 1 . . n] T [1 . . i]$ for an integer $i \in [0 . . n - 1]$ . We say that $T$ and all of its conjugates belong to the conjugate class ${conj} (T) := {{conj}_{0} (T), \dots, {conj}_{n - 1} (T)}$ . If a conjugate class contains exactly one conjugate that is lexicographically smaller than all other conjugates, then this conjugate is called a Lyndon word [20]. Equivalently, a string $T$ is said to be a Lyndon word if and only if $T ≺_{l e x} S$ for every proper suffix $S$ of $T$ [10, Prop. 1.2].

The Lyndon factorization [8] of $T \in Σ^{+}$ is the factorization of $T$ into a sequence of lexicographically non-increasing Lyndon words $T_{1} \dots T_{t}$ , where (a) each $T_{x} \in Σ^{+}$ is a Lyndon word for $x \in [1 . . t]$ , and (b) $T_{x} ⪰_{l e x} T_{x + 1}$ for each $x \in [1 . . t)$ . Each Lyndon word $T_{x}$ is called a Lyndon factor.

[[10, Algo. 2.1]] Given a string $T$ of length $n$ , there is an algorithm that outputs the Lyndon factors $T_{1}, \dots, T_{t}$ one by one in increasing order in $O (n)$ total time while keeping only a constant number of pointers to positions in $T$ that (a) can move one position forward at one time or (b) can be set to the position of another pointer.

Proof.

The algorithm of Duval uses three variables $i$ , $j$ , and $k$ (cf. LABEL:algoDuval in the appendix) pointing to text positions. $k$ is the ending position of the previously computed Lyndon factor (or zero at the beginning). On each step, $j \in [k + 2 . . n]$ is incremented by one, while $i$ is either incremented by one or reset to $k + 1$ , as long as $T [k + 1 . . i - 1] = T [j - (i - k) . . j - 1]$ is a prefix of a Lyndon word starting at $T [k + 1 . .]$ . If $T [k + 1 . . i]$ is no longer such a prefix, then $T [k + 1 . . i - 1]$ is either a Lyndon factor or a repetition of Lyndon factors, each of length $j - i$ . In total, we visit at most $2 n$ characters by incrementing the text positions $i$ , $j$ , and $k$ . ∎

For what follows, we fix a string $T [1 . . n]$ over an alphabet $Σ$ with size $σ$ . We use the string $T := bacabbabb$ as our running example. Its Lyndon factors are $T_{1} = b$ , $T_{2} = ac$ , $T_{3} = abb$ , and $T_{4} = abb$ .

Figure 1: All three BWT variants studied in this paper applied on our running example $T = bacabbabb$ . *Left*: $B B W T$ built on the last characters of the conjugates of all Lyndon words sorted in the $≺_{ω}$ order. *Middle* and *Right*: ${B W T}^{\circ}$ and $B W T$ built on the lexicographically sorted conjugates of $T$ and of $T $$ , respectively. To ease understanding, each character is marked with its position in $T$ in subscript. Reading these positions in $F$ of $B B W T$ and in $F$ of $B W T$ gives a circular suffix array (there are multiple possibilities with $T_{3} = T_{4} = abb$ ) and the suffix array (the position of $ is uniquely defined as $| T $ | = 10$ ).

3.3 Burrows-Wheeler Transforms

We denote the bijective BWT of $T$ by $B B W T$ , where $B B W T [i]$ is the last character of the $i$ -th string in the list storing the conjugates of all Lyndon factors $T_{1}, \dots, T_{t}$ of $T$ sorted with respect to the $≺_{ω}$ order. A property of $B B W T$ used in this paper as a starting point for an inversion algorithm is the following:

[[5, Lemma 15]] $B B W T [1] = T [n]$ .

Proof.

There is no conjugate of a Lyndon factor that is smaller than the smallest Lyndon factor $T_{t}$ since $T_{t} ⪯_{l e x} T_{x} ≺_{l e x} T_{x} [j . .]$ for every $j \in [2 . . | T_{x} |]$ and every $x \in [1 . . t]$ . Therefore, $T_{t}$ is the smallest string among all conjugates of all Lyndon factors. Hence, $B B W T [1]$ is the last character of $T_{t}$ , which is $T [n]$ . ∎

The BWT of $T$ , called in the following $B W T$ , is the BBWT of $$ T$ for a delimiter $$ \notin Σ$ smaller than all other characters in $T$ (cf. [15, Lemma 12] since $$ T$ is a Lyndon word). Originally, the BWT is defined by reading the last characters of all cyclic rotations of $T$ (without $) sorted lexicographically [6]. Here, we call the resulting string ${B W T}^{\circ}$ . ${B W T}^{\circ}$ is equivalent to $B W T$ if $T$ contains the aforementioned unique delimiter $. We further write ${B W T}_{P}$ (and analogously ${B B W T}_{P}$ or ${B W T}_{P}^{\circ}$ ) to denote the BWT of $P$ for a string $P$ .

Since $B W T$ (and analogously $B B W T$ or ${B W T}^{\circ}$ ) is a permutation of $T$ , it is natural to identify each entry of $B W T$ with a text position: By construction $B W T [i] = T [j]$ , where $T [j + 1 . .]$ is the $i$ -th lexicographically smallest suffix, i.e., $S A [i] = j + 1$ , where $S A$ is the suffix array of $T$ . A similar relation is given between $B B W T$ and the circular suffix array [19, 2], which is uniquely defined up to positions of equal Lyndon factors. Figure 1 gives an example for all three variants. In what follows, we review means to simulate a linear traversal of the text in forward or backward manner by $B W T$ , and then translate this result to $B B W T$ .

3.4 Backward and Forward Steps

Having the location of $T [i]$ in $B W T$ , we can compute $T [i + 1]$ (i.e., $T [1]$ for $i = 1$ ) and $T [i - 1]$ (i.e., $T [n]$ for $i = 0$ ) by rank and select queries. To move from $T [i]$ to $T [i + 1]$ , which we call a forward step, we can use the $F L$ mapping:

F L [i] := B W T . {select}_{F [i]} (F . {rank}_{F [i]} (i)),

(1)

where $F [i]$ is the $i$ -th lexicographically smallest character in $B W T$ . To move from $T [i]$ to $T [i - 1]$ , we can use the backward step of the FM-index [11], which is also called $L F$ mapping, and is defined as follows:

L F [i] := F . {select}_{B W T [i]} (B W T . {rank}_{B W T [i]} (i)) = C [B W T [i]] + B W T . {rank}_{B W T [i]} (i),

(2)

where $C [c]$ is the number of occurrences of those characters in $B W T$ that are smaller than $c$ (for each character $c \in [1 . . σ]$ ). We observe from the second equation of (2) that there is no need for $F$ when having $C$ . This is important, as we can compute $C [i]$ in $O (n)$ time only having $B W T$ available. Hence, we can compute $L F [i]$ in $O (n)$ time in-place. However, the same trick does not work with $F L [i] = B W T . {select}_{F [i]} (i - C [F [i]])$ . To lookup $F [i]$ , we can use the selection algorithm of Chan et al. [7] using $B W T$ and $O (lg n)$ bits as working space (the algorithm restores $B W T$ after execution) to compute an entry of $F$ in $O (n)$ time.

In summary, we can compute both $F L [i]$ and $L F [i]$ in-place in $O (n)$ time. The algorithm of Crochemore et al. [9, Thm. 2] inverting $B W T$ in-place in $O (n^{2 + ϵ})$ time uses the result of Munro and Raman [26] computing $F [i]$ in $O (n^{1 + ϵ})$ time for a constant $ϵ > 0$ in the comparison model. As noted by Chan et al. [7, Sect. 1], the time bound for the inversion can be improved to $O (n^{2})$ time in the RAM model under the assumption that $B W T$ is rewritable.

If we allow more space, it is still advantageous to favor storing $C$ instead of $F$ if $σ = o (n)$ because storing $F$ and $C$ in their plain forms take $n lg σ$ bits and $σ lg n$ bits, respectively. To compute $F L [i]$ , we can also compute $F L$ without $F$ by endowing $C$ with a predecessor data structure (which we do in Sect. 4.3).

Finally, we also need $L F$ and $F L$ on $B B W T$ for our conversion algorithms. We can define $L F$ and $F L$ similarly for $B B W T$ with the following peculiarity:

3.5 Steps in the Bijective BWT

The major difference to the BWT is that the LF mapping of the BBWT can contain multiple cycles, meaning that $L F$ (or $F L$ ) recursively applied to a $B B W T$ position would result in searching circular (more precisely, the search stays within the same Lyndon factor). This is because $B B W T$ is the extended BWT [23, Thm. 20 and Remark 12] applied to the multiset of Lyndon factors ${T_{1}, \dots, T_{t}}$

. This fact was exploited for circular pattern matching

[19], but is not of interest here.

Instead, we follow the analysis of the so-called rewindings [3, Sect. 3]: Remembering that we store the last character of all conjugates of all Lyndon factors in $B B W T$ , we observe that the entries in $B B W T$ representing the Lyndon factors (i.e., the last characters of the Lyndon factors) are in sorted order (starting with $T_{t} [| T_{t} |]$ and ending with $T_{1} [| T_{1} |]$ ). That is because the lexicographic order and the $≺_{ω}$ order are the same for Lyndon words [5, Thm. 8]. Applying the backward step at such an entry results in a rewinding, i.e., we can move from the beginning of a Lyndon factor $T_{x}$ (represented by $T_{x} [| T_{x} |]$ in $B B W T$ ) to the end of $T_{x}$ (represented by $T_{x} [1]$ in $B B W T$ ) with one backward step. We use this property with Sect. 3.3 in the following sections to read the Lyndon factors from $T$ individually in the order $T_{t}, \dots, T_{1}$ .

4 Run-Length Compressed Conversions

We now consider $B W T$ and $B B W T$ represented as run-length compressed strings taking $O (r_{B W T} lg n)$ and $O (r_{B B W T} lg n)$ bits of space, where $r_{B W T}$ and $r_{B B W T}$ are the number of character runs in $B W T$ and $B B W T$ , respectively. For $r := max (r_{B W T}, r_{B B W T})$ , the goal of this section is the following:

We can convert $R L B B W T$ to $R L B W T$ in $O (n lg r / lg lg r)$ time using $O (r lg n)$ bits as working space, or vice versa.

To prove this theorem, we need a data structure that works in the run-length compressed space while supporting rank and select queries as well as updates more efficiently than the $O (n)$ time in-place approach described in Sects. 3.5 and 3.4:

4.1 Run-length Compressed Wavelet Trees

Given a run-length compressed string $S$ of uncompressed length $n$ with $r$ character runs, there is an $O (r lg n)$ bits representation of $S$ that supports access, rank, select, insertions, and deletions in $O (lg r)$ time [30, Lemma 1]. It consists of (1) a dynamic wavelet tree maintaining the starting characters of each character run and (2) a dynamic Fenwick tree maintaining the lengths of the runs. It can be accelerated to $O (lg r / lg lg r)$ time by using the following representations:

The dynamic wavelet tree of Navarro and Nekrich [27] on a text of length $r$ uses $O (r lg r)$ bits, and supports both updates and queries in $O (lg r / lg lg r)$ time.
The dynamic Fenwick tree of Bille et al [4, Thm. 2] on $r$ $(lg n)$ -bit numbers uses $O (r lg n)$ bits, and supports both updates and queries in constant time if updates are restricted to be in-/decremental.

The obtained time complexity of this data structure directly improves the construction of $R L B W T$ : [[30, Thm. 2]] We can construct the $R L B W T$ in $O (r_{B W T} lg n)$ bits of space online on the reversed text in $O (n lg r_{B W T} / lg lg r_{B W T})$ time.

In the run-length compressed wavelet tree representation, $R L B W T$ and $R L B B W T$ support an update operation and a backward step in $O (lg r / lg lg r)$ time with $r := max (r_{B W T}, r_{B B W T})$ . This helps us to devise the following two conversions:

4.2 From $R L B B W T$ to $R L B W T$

We aim for directly outputting the characters of $T$ in reversed order since we can then use the algorithm of Sect. 4.1 building $R L B W T$ online on the reversed text. We start with the first entry of $B B W T$ (corresponding to the last Lyndon factor $T_{t}$ , i.e., storing $T_{t} [| T_{t} |] = T [n]$ according to Sect. 3.3) and do a backward step until we come back at this first entry (i.e., we have visited all characters of $T_{t}$ ). During that search, we copy the read characters to $R L B W T$ and mark in an array $R$ of length $r_{B B W T}$ at entry $i$ how often we visited the $i$ -th character run of $R L B B W T$ . Finally, we remove the read cycle of $R L B B W T$ by decreasing the run lengths of $R L B B W T$ by the numbers stored in $R$ . By doing so, we remove the last Lyndon factor $T_{t}$ from $R L B B W T$ and consequently know that the currently first entry of $B B W T$ must correspond to $T_{t - 1}$ . This means that we can apply the algorithm recursively on the remaining $R L B B W T$ to extract and delete the Lyndon factors in reversed order while building $R L B W T$ in the meantime. By removing $T_{t}$ , $B B W T$ is still a valid BBWT since $B B W T$ becomes the BBWT of $T^{'} := T_{1} \dots T_{t - 1}$ whose Lyndon factors are the same as of $T$ (but without $T_{t}$ ). Note that it is also possible to build $R L B W T$ in forward order, i.e., computing ${R L B W T}_{T_{1} \dots T_{x}}$ for increasing $x$ by applying the algorithm of Mantaci et al. [24, Fig. 1] while omitting the suffix array construction.

4.3 From $R L B W T$ to $R L B B W T$

To build $B B W T$ , we need to be aware of the Lyndon factors of $T$ , which we compute with Sect. 3.2 by simulating a forward scan on $T$ with $F L$ on $B W T$ . To this end, we store the entries of the $C$ array in a Fusion tree [13] using $O (σ lg n)$ bits and supporting predecessor search in $O (lg σ / lg lg σ) = O (lg r / lg lg r)$ time.²²2We assume that the alphabet $Σ$ is effective, i.e., that each character of $Σ$ appears at least once in $T$ . Otherwise, assume that $T$ uses $σ^{'}$ characters. Then we build the static dictionary of Hagerup [18] in $O (σ^{'} lg σ^{'})$ time, supporting access to a character in $O (lg lg σ^{'}) = O (lg lg r)$ time, assigning each of the $σ^{'}$ characters an integer from $[1 . . σ^{'}]$ . We further map $R L B W T$ to the alphabet $[1 . . σ^{'}]$ , which can be done in $O (r)$ time by using $O (r lg n)$ space for a linear-time integer sorting algorithm. This time complexity also covers a forward step in $R L B W T$ by simulating $F$ with the Fusion tree on $C$ . Hence, this fusion tree allows us to apply Sect. 3.2 computing the Lyndon factorization of $T$ with a multiplicative $O (lg r / lg lg r)$ time penalty since this algorithm only needs to perform forward traversals. The starting point of such a traversal is the position $i$ with $B W T [i] = $$ because $F L [i]$ returns the first character of $T$ . Whenever we detect a Lyndon factor $T_{x}$ (starting with $x = 1$ ), we copy this factor to our dynamic $R L B B W T$ . For that, we always maintain the first and the last position of $T_{x}$ in memory. Having the last position of $T_{x}$ , we perform backward steps on $R L B W T$ until returning at the first position of $T_{x}$ to read the characters of $T_{x}$ in reversed order. Then we continue with the algorithm of Sect. 3.2 at the position after $T_{x}$ (for recursing on $T_{x + 1}$ ). Inserting a Lyndon factor into $R L B B W T$ works exactly as sketched by Bonomo et al. [5, Thm. 17] or in LABEL:algoConstructBBWT in the appendix (we review this algorithm in detail in Sect. 5.1).

5 In-Place Conversions

We finally present our in-place conversions that work in quadratic time by computing $L F$ or $F L$ in $O (n)$ time having only stored either $B W T$ , $B B W T$ , or ${B W T}^{\circ}$ . We note that the constructions from the text also work in the comparison model, while inverting a transform or converting two different transforms have a multiplicative $O (n^{ϵ})$ time penalty as the fastest option to access $F$ in the comparison model uses $O (n^{1 + ϵ})$ time for a constant $ϵ > 0$ [26]. We start with the construction and inversion of ${B W T}^{\circ}$ (Sects. 5.2 and 5.1), where we show (a) that we can construct ${B W T}^{\circ}$ from the text in the same manner as Bonomo et al. [5] construct $B B W T$ , and (b) that the latter construction works also in-place. Next, we show in Sect. 5.3 how to invert $B B W T$ with the BWT inversion algorithm of Crochemore et al. [9, Fig. 3], which allows us to also convert $B B W T$ to $B W T$ with the BWT construction algorithm of the same paper [9, Fig. 2]. Finally, we show a conversion from $B W T$ to $B B W T$ in Sect. 5.4. An overview is given in Table 1.

ToFrom $T$ $B W T$ $B B W T$ ${B W T}^{\circ}$ $T$ \ [9, Fig. 3] Sect. 5.3 Sect. 5.2 $B W T$ [9, Fig. 2] \ Sect. 5.3 $B B W T$ Sect. 5.1 Sect. 5.4 \ ${B W T}^{\circ}$ Sect. 5.1 \

Table 1: Overview of in-place conversions in focus of Sect. 5 working in quadratic time.

5.1 Constructing ${B W T}^{\circ}$ and $B B W T$

We can compute ${B W T}^{\circ}$ and $B B W T$ from $T$ with the algorithm of Bonomo et al. [5] computing the extended BWT [23]. The extended BWT is the BWT defined on a set of primitive strings. As stated in Sect. 3.5, the extended BWT coincides with $B B W T$ if this set of primitive strings is the set of Lyndon factors of $T$ [5, Thm. 14]. We briefly describe the algorithm of Bonomo et al. [5] for computing the BBWT (cf. Fig. 2 and LABEL:algoConstructBBWT in the appendix): For each Lyndon factor $T_{x}$ (starting with $x = 1$ ), prepend $T_{x} [| T_{x} |]$ to $B B W T$ . To insert the remaining characters of the factor $T_{x}$ , let $p \leftarrow 1$ be the position of the currently inserted character. Then perform, for each $j = | T_{x} | - 1$ down to $1$ , a backward step $p \leftarrow L F [p] + 1$ , and insert $T_{x} [j]$ at $B B W T [p]$ (cf. LABEL:algoConstructBBWT in the appendix). To understand why this computes $B B W T$ , we observe that the last character of the most recently inserted Lyndon factor $T_{x}$ is always the first character in ${B B W T}_{T_{1} \dots T_{x}}$ according to Sect. 3.3. By recursively inserting the preceding character at the place returned by a backward step, we precisely insert this character at the position where we would expect it (another backward step from the same position $p$ would then return the inserted character). Using only $n$ backward steps and $n$ insertions, this algorithm works in-place in $O (n^{2})$ time by simulating $L F$ as described in Sect. 3.4.

Consequently, we can build ${B W T}^{\circ}$ if $T$ is a Lyndon word since in this case ${B W T}^{\circ}$ and $B B W T$ coincide [15, Lemma 12]. That is because sorting the suffixes of $T$ is equivalent to sorting the conjugates of $T$ (if $T$ is a Lyndon word, then its Lyndon factorization consists only of $T$ itself).

It is easy to generalize this to work for a general string $T$ . First, if $T$ is primitive, then we compute its so-called Lyndon conjugate, i.e., a conjugate of $T$ that is a Lyndon word. (The Lyndon conjugate of $T$ is uniquely defined if $T$ is primitive.) We can find the Lyndon conjugate of $T$ in $O (n)$ time with the following two lemmata:

[ [10, Prop. 1.3] ] Given two Lyndon words $S$ and $T$ , $S T$ is a Lyndon word if $S ≺_{l e x} T$ .

Given a primitive string $T$ , we can find its Lyndon conjugate in $O (n)$ time with $O (lg n)$ bits of space.

Proof.

We use Sect. 3.2 to detect the last Lyndon factor $T_{t}$ of the Lyndon factorization $T_{1} \dots T_{t}$ of $T$ with $O (lg n)$ bits of working space. According to Sect. 5.1, $T_{t} T_{1}$ is a Lyndon word since $T_{t} ≺_{l e x} T_{1}$ , and so is $T_{t} T_{1} \dots T_{t - 1}$ a Lyndon word by a recursive argument. Hence, we have found $T$ ’s Lyndon conjugate. ∎

Let ${conj}_{j} (T)$ be the Lyndon conjugate of $T$ for $j \in [0 . . n - 1]$ . Since ${B W T}^{\circ}$ is identical to ${B B W T}_{{conj}_{j} (T)}$ , we are done by running the algorithm of Bonomo et al. [5] on ${conj}_{j} (T)$ . Finally, if $T$ is not primitive, then there is a primitive string $P$ such that $T = P^{k}$ for an integer $k \geq 2$ . We can compute ${B W T}_{P}^{\circ}$ with the above considerations. For obtaining ${B W T}^{\circ}$ , according to [25, Prop. 2], we only need to make each character in ${B W T}_{P}^{\circ}$ to a character run of length $k$ , i.e., if ${B W T}_{P}^{\circ} [i] = c$ , we append $c^{k}$ to ${B W T}^{\circ}$ for increasing $i \in [1 . . | P |]$ (cf. [15, Thm. 13]). Checking whether $T$ is primitive can be done in $O (n^{2})$ time by checking for each pair of positions their longest common prefix. We summarized these steps in the pseudo code of LABEL:algoConstructBWTC in the appendix.

Figure 2: Computing $B B W T$ from our running example $T = bacabbabb$ in four steps (visualized by four columns separated by three arrows ), cf. Sect. 5.1. In each column, the characters from the top to the solid horizontal line ( ) form the currently built BBWT. The characters below that up to the dashed horizontal line ( ) are under consideration of being merged into BBWT. This dashed line is always before the beginning of the next yet unread Lyndon factor. *First column*: We have already computed the BBWT of $T_{1} T_{2} = bac$ , which is cba. In the following we want to add the next Lyndon factor $T_{3} = abb$ to it. For that, we prepend its last character to the currently constructed BBWT. *Second column*: We move the last character above the dashed line to the position $\definecolor[named]pgfstrokecolorrgb0.796875,0.29296875,0.0859375\pgfsys@color@rgb@stroke0.7968750.292968750.0859375\pgfsys@color@rgb@fill0.7968750.292968750.0859375LF[\definecolor[named]pgfstrokecolorrgb0.5234375,0.6015625,0\pgfsys@color@rgb@stroke0.52343750.60156250\pgfsys@color@rgb@fill0.52343750.60156250p]+1$ with $\definecolor[named]pgfstrokecolorrgb0.5234375,0.6015625,0\pgfsys@color@rgb@stroke0.52343750.60156250\pgfsys@color@rgb@fill0.52343750.60156250p=1$ , and update . We recurse in the *third column*, and have produced the BBWT of $T_{1} T_{2} T_{3} = bacabb$ in the *forth column*.

5.2 Inverting ${B W T}^{\circ}$

To invert ${B W T}^{\circ}$ , we use the techniques of Crochemore et al. [9, Fig. 3] inverting $B W T$ in-place in $O (n^{2})$ time. An invariant is that the BWT entry, whose $F L$ mapping corresponds to the next character to output, is marked with a unique delimiter $. Given that $B W T [i] = $$ , the algorithm outputs $B W T [F L [i]]$ , sets $B W T [F L [i]] \leftarrow $$ , removes $B W T [i]$ , and recurses until $ is the last character remaining in $B W T$ . By doing so, it restores the text in text order.

To adapt this algorithm for inverting ${B W T}^{\circ}$ , we additionally need a pointer $p$ storing the first symbol of the text (since there is no unique delimiter such as $ in general). Given that $p$ points to ${B W T}^{\circ} [i]$ , we set $i \leftarrow F L [i]$ and subsequently output ${B W T}^{\circ} [i]$ . From now on, the algorithm works exactly as [9, Fig. 3] if we set ${B W T}^{\circ} [i] \leftarrow $$ after outputting ${B W T}^{\circ} [i]$ (cf. LABEL:algoInvertBWTC in the appendix). More involving is inverting $B B W T$ or converting $B B W T$ to $B W T$ , which we tackle next.

Figure 3: Inverting $B B W T$ of our running example $T = bacabbabb$ (cf. Sect. 5.3). *First Column*: We prepend the $ delimiter to the last Lyndon factor $T_{t}$ by inserting $ at $B B W T [2]$ . A forward step symbolized by the dashed arrow ( ) leads us from $ to the first character of $T_{t}$ . *Second Column*: We output $B B W T [6] = T_{t} [1] = T [7]$ , remove $ and update $B B W T [6] \leftarrow $$ . The output is appended to the string shown below the dashed horizontal line ( ). We continue with a forward step to access $B B W T [4] = T_{t} [2] = T [8]$ , and recurse in the *third column*. *Forth Column*: Since a forward step returns $, we know that we have successfully extracted $T_{t} = abb$ .

5.3 Inverting $B B W T$

Similarly to Sect. 4.2, we read the Lyndon factors from $B B W T$ in the order $T_{t}, \dots, T_{1}$ , and move each read Lyndon factor directly to a text buffer such that while reading the last Lyndon factor $T_{x}$ for an $x \in [1 . . t]$ from ${B B W T}_{T_{1} \dots T_{x}}$ , we move the characters of $T_{x}$ to $T_{x + 1} \dots T_{t}$ , producing ${B B W T}_{T_{1} \dots T_{x - 1}}$ and $T_{x} \dots T_{t}$ . This allows us to recurse by reading always the last Lyndon factor $T_{x}$ stored in ${B B W T}_{T_{1} \dots T_{x}}$ .

Here, we want to apply the inversion algorithm for ${B W T}^{\circ}$ described in Sect. 5.3. For adapting this algorithm to work with $B B W T$ , it suffices to insert $ at $B B W T [2]$ (cf. Fig. 3). By doing so, we add $ to the cycle of the currently last Lyndon factor $T_{x}$ stored in $B B W T$ , i.e., we enlarge the Lyndon factor $T_{x}$ to $$ T_{x}$ . That is because (a) $B B W T [1]$ corresponds to the last character $T_{x} [| T_{x} |]$ of $T_{x}$ (cf. Sect. 3.3), and after inserting $, $F [1] = $, F [2] = T_{x} [1]$ , hence $F L [1] = 2$ (a forward step on the last character of $T_{x}$ gives $) and $F L [2]$ gives the position in $B B W T$ corresponding to $T_{x} [1]$ . Moreover, inserting $ makes $B B W T$ the BBWT of $T^{'} := T_{1} \dots T_{x_{1}} $ T_{x}$ , where $$ T_{x}$ is the last Lyndon factor of $T^{'}$ . We now use the property that $$ T_{x} [i . . | T_{x} |]$ is a Lyndon word for each $i \in [1 . . | T_{x} |]$ , allowing us to perform the inversion steps of Crochemore et al. [9, Fig. 3] on $B B W T$ . By doing so, we can remove the entry of $B B W T$ corresponding to ${conj}_{j} (T_{x})$ for increasing $j \in [0 . . | T_{x} | - 1]$ and prepend the extracted characters to the text buffer storing $T_{x + 1} \dots T_{t}$ within our working space while keeping $B B W T$ a valid BBWT.

Instead of inverting $B B W T$ , we can convert $B B W T$ to $B W T$ in-place by running the in-place BWT construction algorithm of Crochemore et al. [9, Fig. 2] on the text buffer after the extraction of each Lyndon factor. Unfortunately, this works not character-wise, but needs a Lyndon factor to be fully extracted before inserting its characters into $B W T$ . Interestingly, for the other direction (from $B W T$ to $B B W T$ ), we can propose a different kind of conversion that works directly on $B W T$ without decoding it.

5.4 From $B W T$ to $B B W T$ on the Fly

Like in Sect. 4.3, we process the Lyndon factors of $T$ individually to compute $B B W T$ by scanning $B W T$ in text order to simulate Sect. 3.2. Suppose that we have built $B W T$ on $T $ \neq $$ with $ being the $(t + 1)$ -th Lyndon factor of $T $$ , and suppose that we have detected the first Lyndon factor $T_{1}$ . Let f denote the last character of $T_{1}$ .³³3f stands for final character. Further let $i_{f}$ and $i_{$}$ be the position of the last character of $T_{1}$ and the last character of $T$ , respectively, such that $B W T [i_{f}] = f$ and $B W T [i_{$}] = $$ . Let $p := L F [i_{f}]$ such that $F [p] = f$ and $B W T [p] = T_{1} [| T_{1} | - 1]$ if $| T_{1} | > 1$ or $B W T [p] = $$ otherwise. Since $T_{1}$ and $T_{2}$ are Lyndon factors, $T_{1} ⪰_{l e x} T_{2}$ . Consequently, the suffix $T [{b} (T_{2}) . .]$ (the context of $B W T [i_{f}]$ ) is lexicographically smaller than the suffix $T [{b} (T_{1}) . .]$ (the context of $B W T [i_{$}]$ ), i.e., $i_{f} < i_{$}$ . Figure 4 gives an overview of the introduced setting.

Figure 4: Setting of Sect. 5.4 with focus on forming a cycle for a Lyndon factor ending with f in $B W T$ . *Left*: We exchange $B W T [i_{f}]$ with $B W T [i_{$}]$ with the aim to form a cycle. *Right*: To obtain this cycle we additionally need to swap $B W T [p]$ with the elements of the dashed rectangle ( ) corresponding to the interval $I$ having the same height as the dotted rectangle ( ) covering $B W T [i_{f} + 1 . . i_{$} - 1]$ .

Figure 5: Computing $B B W T$ from $B W T$ (cf. Sect. 5.4) of our running example $T = bacabbabb{}$$ . In the *left* column, we find the first Lyndon factor $T_{1} = b$ of $T$ by forward steps with $F L$ . Since $| T_{1} | = 1$ , $p = i_{$}$ . We obtain the *middle* column by exchanging $B W T [4]$ with $B W T [7] = $$ . Since there are two b’s between b at $B W T [4]$ and $ in the *left* column, we need to swap $B W T [p]$ with the two elements below of it in the *middle* column. This gives a cycle in the *right* column. We can recurse since the $F L$ mapping of $ now yields the second character of $T$ .

Figure 6: Special case for computing $B W T$ from $B B W T$ (cf. Sect. 5.4) with the different example string $T $ := cedabedad$$ having $T_{1} = ced$ as its first Lyndon factor. *Left column*: We find the first Lyndon factor $T_{1} = ced$ of $T$ by forward steps with $F L$ . Its last character is stored at $B W T [2]$ . By exchanging $ with the last character of $T_{1}$ in $B W T$ , we obtain the middle column. *Middle column*: The $L F$ mapping for the third d in $F$ becomes invalid. However, there is only a character run of $T_{1} [| T_{1} | - 1] = e$ in $B W T$ of the $T_{1} [| T_{1} |] = d$ interval $[7 . . 8]$ in $F$ starting with $p = 7$ . So we recurse on $L F [p]$ to find characters different from $T_{1} [| T_{1} | - 2] = c$ to swap in the respective $T_{1} [| T_{1} | - 1] = e$ interval $[9 . . 10]$ . *Right Column*: We have created a cycle with the characters of the first Lyndon factor. A forward step at $ gives the first character of the next Lyndon factor.

Our aim is to change $B W T$ such that a forward or backward step within the characters belonging to $T_{1}$ always results in a cycle. Informally, we want to cut $T_{1}$ out of $B W T$ , which additionally allows us to recursively continue with the $F L$ mapping to find the end of the next Lyndon factor $T_{2}$ .⁴⁴4As a matter of fact, if we now want to restore the text with the modified $B W T$ by $L F$ , we would only produce $T_{2} \dots T_{t}$ . For that, we exchange $B W T [i_{$}]$ with $B W T [i_{f}]$ (cf. Fig. 5). Then the character $T [{e} (T_{1}) + 1]$ (i.e., the first character of $T_{2}$ ) becomes the next character of $ in terms of the forward step ( $B W T [F L [i_{f}]] = T [{b} (T_{2})]$ ), while a backwards search on the first character of $T_{1}$ yields $T_{1}$ ’s last character ( $L F$ returns $i_{$}$ , but now $B W T [i_{$}] = T_{1} [| T_{1} |] = f$ ). This is sufficient as long as $B W T [i] \neq f$ for every $i \in (i_{f} . . i_{$}]$ . Otherwise, it can happen that we change the mapping from the $i$ -th f of $F$ to the $i$ -th f of $B W T$ (or vice versa) unintentionally. In such a case, we swap some entries in $B W T$ within the f interval of $F$ . In detail, we conduct the exchange ( $B W T [i_{$}]$ with $B W T [i_{f}]$ ), but continue with swapping $B W T [i]$ and $B W T [i + 1]$ unless $B W T [F L [i]]$ becomes that f that corresponds to $T_{1} [| T_{1} |]$ for increasing $i$ starting with $i = p$ until $F [i] \neq f$ or $L F [i] \notin [i_{f} . . i_{$}]$ . This may not be sufficient if the characters we swap are identical (cf. Fig. 6). In such a case, we recurse on the $T_{1} [| T_{1} | - 1]$ interval of $F$ , see also LABEL:algoBWTToBBWT in the appendix.

Instead of checking whether we have created a cycle after each swap, we want to compute the exact number of swaps needed for this task. For that we note that exchanging $B W T [i_{$}]$ with $B W T [i_{f}]$ decrements the values of $B W T . {rank}_{f} (j)$ for every $j \in [i_{f} . . i_{$}]$ by one. In particular, $B W T . {select}_{f}$ changes for those f’s in $B W T$ that are between $i_{f}$ and $i_{$}$ . Hence, the number of swaps $m$ is the number of positions $k \in [i_{f} + 1 . . i_{$} - 1]$ with $B W T [k] = f$ . The swaps are performed within the range $I$ starting with $p + 1$ and covering all positions $i$ with $L F [i] \in [i_{i} . . i_{$}]$ and $F [i] = f$ since $I$ covers all entries whose mapping has changed. However, if $B W T [p . .]$ starts with a character run of $T [{e} (T_{1}) - 1]$ (or of $T [{b} (T_{1})]$ if $| T_{1} | = 1$ )⁵⁵5For $| T_{1} | = 1$ , $p = i_{$}$ , and hence, $B W T [p]$ was $ but now is $f = T_{1} [| T_{1} |] = T_{1} [1]$ ., swapping the identical characters does not change $B W T$ , and therefore has no effect of changing $L F$ . Instead, we search the end of this run within $I$ to swap the first entry $i$ below this run with the first entry of this run, and recurse on swapping entry $i$ with entries below of it.

Correctness.

To see why the swaps restore the LF mapping for $T_{1}$ and the remaining part of the text $T_{2} \dots T_{t}$ , we examine those substrings of $T$ that we might no longer find with the LF mapping after exchanging $B W T [i_{$}]$ with $B W T [i_{f}]$ .

In detail, we examine each substring $S_{j} := x_{j} y_{j} f \in Σ^{3}$ with $j \in [1 . . m]$ that is represented in $B W T$ (before changing it) with $B W T [p + j] = y_{j}, B W T [L F [p + j]] = x_{j}, B W T [F L [p + j]] = f$ , and $i_{j} := F L [p + j] \in [i_{f} + 1 . . i_{$} - 1]$ . Due to the LF-mapping, $B W T . {select}_{f} (B W T . {rank}_{f} (i_{f}) + j) = i_{j}$ , meaning that $B W T [i_{j}]$ is the $j$ -th f in $B B W T [i_{f} + 1 . . i_{$} - 1]$ , which stores $m$ f’s. After exchanging $B W T [i_{$}]$ with $B W T [i_{f}]$ , $F L [p + j]$ becomes $i_{j + 1}$ for $j \in [0 . . m]$ with $i_{m + 1} := i_{$}$ . However, for all $i > p + m$ , $F L [i]$ did not change. Hence, we only have to focus on the range $I = [p + 1 . . p + m]$ .

First, suppose that $y_{1} = B W T [p + 1] \neq B W T [p]$ . If we swap $B W T [p]$ with $B W T [p + 1]$ , then $L F [p]$ is still $i_{1}$ , but $B W T [L F [p]]$ becomes $x_{1}$ such that we have fixed the substring $x_{1} y_{1} f$ . This also works in a more general setting: If $y_{j} = B W T [p + j] \neq B W T [p]$ for every $j \in [1 . . m]$ , we can perform $m$ swaps like above for all $m$ entries in $B W$

In-Place Bijective Burrows-Wheeler Transforms

Authors

Efficient Representation and Counting of Antipower Factors in Words

Strictly In-Place Algorithms for Permuting and Inverting Permutations

Resolution of the Burrows-Wheeler Transform Conjecture

MONI can find k-MEMs

An in-place truncated Fourier transform

Local Decodability of the Burrows-Wheeler Transform

An in-place, subquadratic algorithm for permutation inversion

1 Introduction

2 Related Work