Bounds on the Error Probability of Raptor Codes under Maximum Likelihood Decoding

09/05/2018 ∙ by Francisco Lázaro, et al. ∙ DLR University of Bologna 0

In this paper upper bounds on the probability of decoding failure under maximum likelihood decoding are derived for different (non-binary) Raptor code constructions. In particular four different constructions are considered; (i) the standard Raptor code construction, (ii) a multi-edge type construction, (iii) a construction where the Raptor code is non-binary but the generator matrix of the LT code has only binary entries, (iv) a combination of (ii) and (iii). The latter construction resembles the RaptorQ construction, which at the time of writing this article represents the state of the art in fountain codes. The bounds are shown to be tight, and provide an important aid for the design of Raptor codes.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Fountain codes [2] are a class of erasure codes which have the property of being rateless. Thus, they are potentially able to generate an endless amount of encoded (or output) symbols from information (or input) symbols. This property makes them suitable for application in situations where the channel erasure rate is not a priori known. The first class of practical fountain codes, Luby Transform (LT) codes, was introduced in [3] together with an iterative decoding algorithm that achieves a good performance when the number of input symbols is large. In [3, 4] it was shown how, in order to achieve a low probability of decoding error, the encoding and iterative decoding cost111In [4] the cost per output symbol is defined as the encoding/decoding complexity normalized by the number of output symbols. The complexity is defined as the number operations needed to carry out encoding/decoding. per output symbol is .

Raptor codes were introduced in [4] and outperform LT codes in several aspects. They consist of a serial concatenation of an outer code (or precode) with an inner LT code. On erasure channels, this construction allows relaxing the design of the LT code, requiring only the recovery of a fraction of the input symbols, with small. This can be achieved with linear encoding and decoding complexity (under iterative decoding). The outer code is responsible for recovering the remaining fraction of input symbols. If the outer code is linear-time encodable and decodable then the Raptor code has linear encoding and (iterative) decoding complexity over erasure channels.

Most of the existing works on LT and Raptor codes consider iterative decoding and assume large input block lengths ( at least in the order of a few tens of thousands). However, in practice, smaller values of are more commonly used. For example, for the binary Raptor codes standardized in [5] and [6] the recommended values of range from to . For these input block lengths, iterative decoding performance degrades considerably. In this regime, a different decoding algorithm may be adopted that is an efficient maximum likelihood (ML) decoder, in the form of inactivation decoding [7, 8, 9, 10, 11].

An inactivation decoder solves a system of equations in several stages. First a set of variables is declared to be inactive. Next a system of equations involving only the set of inactive variables needs to be solved, for example using Gaussian elimination. Finally, once the value of the inactive variables is known, all other variables (those which were not inactive) are recovered using iterative decoding (back substitution).

Recently, several works have addressed the complexity of inactivation decoding for Raptor and LT codes [12, 13, 14, 15]. The probability of decoding failure of LT and Raptor codes under ML decoding has also been subject of study in several works. In [16] upper and lower bounds on the symbol erasure rate were derived for LT codes and Raptor codes with outer codes in which the elements of the parity check matrix are independent and identically distributed (i.i.d.

) Bernoulli random variables. This work was extended in

[17], where upper and lower bounds on the performance of LT codes under ML decoding were derived. A further extension was presented in [18], where an approximation to the performance of Raptor codes under ML decoding is derived under the assumption that the number of erasures correctable by the outer code is small. Hence, this approximation holds when the rate of the outer code is sufficiently high. In [19] it was shown by means of simulations how the error probability of -ary Raptor codes is very close to that of linear random fountain codes. In [20] upper and lower bounds on the probability of decoding failure of Raptor codes were derived. The outer codes considered in [20] are binary linear random codes with a systematic encoder. Ensembles of Raptor codes with linear random outer codes were also studied in a fixed-rate setting in [21],[22]. In [23], -ary Raptor codes are considered (i.e., Raptor codes constructed over a finite field of order , ), but only for the case in which the outer code is a low-density generator matrix code. Although a number of works has studied the probability of decoding failure of Raptor codes, to the best of the authors’ knowledge, up to now the results hold only for specific outer codes (see [16, 20, 21, 22, 23]).

In this paper upper bounds on the probability of decoding failure of different Raptor code constructions are derived. In contrast to other works in literature [16, 20, 21, 22, 23], the bounds presented in this paper are general since they are valid for any outer code, requiring only the weight enumerator (or composition enumerator, a quantity to be defined later) of the outer code. Furthermore, simulation results are presented which show how the derived bounds are tight. In particular four different constructions are considered, namely:

  1. [label=)]

  2. a Raptor code construction over , where the outer code is built over as well as the generator matrix of the LT code;

  3. a multi-edge type Raptor construction over , where intermediate symbols of two different types can be distinguished;

  4. a construction where the Raptor code is built over but the generator matrix of the LT code has only entries belonging to ;

  5. a combination of (ii) and (iii).

The bounds are applicable for the two Raptor codes present in standards. In particular, the R10 Raptor code in its nonsystematic form [5] is an example of construction (i) since binary Raptor codes are simply a special case (). Furthermore, the RaptorQ code in its nonsystematic form [24] is an example of construction (iv). The RaptorQ code is, at the timing of writing this article, the state of the art fountain code construction, and it is an IETF standard [24]. To the best of the authors’ knowledge this is the first work which is analyzes the performance of the RaptorQ construction222In [23] a -ary Raptor code construction is analyzed, but it does not consider all the peculiarities of the RaptorQ code.

The paper is organized as follows. In Section II some preliminary definitions are given. Section III addresses the different Raptor code constructions considered in this paper. Section IV presents several theorems with upper bounds on the probability of decoding failure for the different Raptor code constructions. Section V contains the proofs of the results presented in this paper. Numerical results are illustrated in Section VI. Section VII presents the conclusions of our work.

Ii Preliminaries

For any linear block code constructed over and any codeword , we let be the Hamming weight (often referred to simply as the weight) of . Letting be the codeword length, we shall denote the weight enumerator of as , where denotes the multiplicity of codewords of weight . Similarly, given an ensemble of linear block codes, all with the same block length

, along with a probability distribution on the codes in the ensemble, we shall denote the expected weight enumerator of a random code in

as , where denotes the expected multiplicity of codewords of weight for the random code.

Next, consider a linear block code , whose codeword symbols are partitioned into two different types, namely, type and type . Let and be the number of codeword symbols of types and , respectively, such that . A generic codeword after reordering can be expressed as , where and

denote the vectors of encoded symbols of type

and type respectively. In this context the bivariate weight enumerator polynomial of the code is defined as

(1)

where denotes the multiplicity of codewords with and . Similarly, given an ensemble of block codes with block length and with two types of codeword symbols as defined above, along with a probability distribution on the codes in the ensemble, we shall define its expected bivariate weight enumerator polynomial as

where denotes the expected multiplicity of codewords with and .

Given a vector , we define its composition as

where

being the residue class of the polynomial , and

Given a linear block code , we define its composition enumerator, , as the number of codewords with composition . Similarly, for a code ensemble we define its expected composition enumerator as the expected multiplicity of codewords with composition .

Consider also a linear block code of length , with two types of codeword symbols as defined above. We define the bivariate composition enumerator of a code as the number of codewords in for which has composition and has composition . This definition can be easily extended to code ensembles. In particular, we define the expected bivariate composition enumerator of a random code in the ensemble as the expected multiplicity of codewords for which has composition and has composition for the random code.

Given the composition of a vector , , as defined above, we define as an indicator function that takes value only if , i.e.,

(2)

Moreover, for a positive integer and a prime or prime power , we denote by the Krawtchouk polynomial of degree with parameters and , which is defined as [25]

Finally, we recall the expression of the Chu-Vandermonde identity, stating that

Iii Raptor Codes

Iii-a Encoding and Decoding

We consider four different Raptor code constructions, all of them over , with , being a prime or prime power. Fig. 1 shows a block diagram of Raptor encoding. In particular we consider an outer linear block code whose length and dimension are denoted by and , respectively. We denote the input (or source) symbols of the Raptor code as . Out of the input symbols, the outer code generates a vector of intermediate symbols . Denoting by the generator matrix of the outer code, of dimension , the intermediate symbol vector can be expressed as

The intermediate symbols serve as input to an LT encoder, which generates the output symbols , where can grow unbounded. For any the output symbol vector can be expressed as

(3)

where is an matrix. The different constructions addressed in this paper differ in how matrix is built, as we will explain later in this section.

The output symbols are transmitted over a -ary erasure channel (-EC). At its output each transmitted symbol is either correctly received or erased.333We remark that, due to the fact that LT output symbols are generated independently of each other, the results developed in this paper remain valid regardless the statistic of the erasures introduced by the channel. We denote by the number of output symbols collected by the receiver, and we express it as , where is the absolute receiver overhead. Let us denote by the vector of received output symbols. Denoting by the set of indices corresponding to the non-erased symbols, we have

An ML decoder proceeds by solving the linear system of equations

(4)

where

(5)

and where is the submatrix of formed by the columns with indices in .

Figure 1: Block diagram of Raptor encoding.

Iii-B Raptor Code Constructions

The first construction considered in this paper shall be referred to as Raptor code over . In this construction each column of is generated by first selecting an output degree , according to a probability distribution , and then by selecting different indices uniformly at random between and . The distribution is usually referred to as output degree distribution and its generating function is

Finally, the elements of the column in the row positions corresponding to these indices are drawn independently and uniformly at random from , while all other elements of the column are set to zero.

The second considered construction shall be referred to as multi-edge type Raptor code. This construction is characterized by having two different types of intermediate symbols, namely, type and type . Thus, the vector of intermediate symbols after reordering can be expressed as , where and denote the vectors of intermediate symbols of types and respectively. Furthermore, we denote the number of intermediate symbols of type and as and respectively. We have . This Raptor code construction is characterized by a relationship between output symbols and intermediate symbols in the form

(6)

Under the assumption that output symbols are generated, and have sizes and respectively. Each column of is generated by first selecting two output degrees and according to a joint probability distribution whose bivariate generating function is444This definition implies (besides ), which is in line with the distribution used for the RaptorQ code [24]. This assumption is practically motivated but is not strictly necessary.

For each column, different indices are picked uniformly at random in and the elements of the column in at the rows corresponding to these indices are drawn independently and uniformly from , while all other elements of the column of are set to zero. In a similar way, different indices are picked uniformly at random in and the elements of the column in at the rows corresponding to these indices are drawn independently and uniformly from , while all other elements of the column of are set to zero.

The third construction considered shall be referred to as Raptor code over with a LT code. This construction is relevant to , since otherwise it collapses to the first construction. It is similar to the first construction (Raptor code over ), but all non-zero coefficients of are equal to . Thus, each column of is generated by first selecting an output degree according to the degree distribution , and then by selecting different indices uniformly at random in . Finally, the elements of the column with rows corresponding to these indices are set to , while all other elements of the column are set to zero. The relationship between input and output symbols is still given by (3), where vectors , and have elements in , matrix has elements in as well, and the elements of belong to . The advantage of this construction is that encoding and decoding complexities are significantly reduced when using a standard computing platform, particularly when is a power of .

Finally, the fourth construction considered is referred to as multi-edge type Raptor code over with a LT code. As its name indicates this construction is a combination of the second and third constructions described before. In particular, this construction is the same as the second construction, except for the fact that the non-zero elements in , and therefore in and , are not selected uniformly at random from but take always value .

This last construction closely resembles the RaptorQ code [24], representing the state of art fountain code at the time of writing. The RaptorQ code is built over . Its outer code is itself obtained as the serial concatenation of two block codes, the outer code being a quasi-cyclic non-binary LDPC code and a the inner code being a non-binary high-density parity check (HDPC) code. In particular, the quasi-cyclic LDPC code has all its nonzero elements in the parity check matrix equal to , whereas the HDPC code resembles a random code over . The intermediate symbols belong to two different classes, which are called LT symbols and permanently inactive symbols. The LT code is a LT code characterized by the bivariate degree distribution

where and

are, respectively, the dummy variables associated with LT and permanently inactive symbols, and

is a degree distribution with maximum output degree . Finally, we remark that the RaptorQ construction can be made systematic.555A Raptor code is made systematic by adding a further precoding stage and specifying the seed of the pseudorandom generator which is used to generate the LT output symbols, see [26, 27] for more details Thus, the RaptorQ code in its non-systematic form666The RaptorQ code is in non-systematic form when random Encoding Symbol Identifiers (ESI) are used [28]. is an example of the fourth construction considered in this paper (multi-edge type Raptor code over with a LT code). For more details about the RaptorQ construction as well as the design choices involved we refer the reader to [29].

Iv Upper Bounds on the Error Probability
of Raptor Codes

This section contains the main contribution of this paper, a series of bounds on the performance of the different Raptor code constructions presented in Section III. Proofs of these bounds are deferred to Section V. The first theorem establishes a bound on the probability of decoding failure of a Raptor code over .

Theorem 1.

Consider a Raptor code over with an outer code characterized by a weight enumerator , and an inner LT code with output degree distribution . The probability of decoding failure under ML erasure decoding, given that output symbols have been collected by the receiver, can be upper bounded as

where is the probability that the generic output symbol is equal to given that the vector of intermediate symbols has Hamming weight . The expression of is

(7)

The upper bound in Theorem 1 also applies to LT codes. In that case, and is simply the total number of sequences of Hamming weight and length ,

The upper bound thus obtained for LT codes coincides with the bound in [17, Theorem 1].

Theorem 1 may be extended to multi-edge type Raptor codes over as follows.

Theorem 2.

Consider a multi-edge type Raptor code over with an outer code characterized by a bivariate weight enumerator polynomial and an inner LT code with bivariate output degree distribution . The probability of decoding failure under ML erasure decoding given that output symbols have been collected by the receiver can be upper bounded as

where

(8)

The next result establishes a bound on the probability of decoding failure of a Raptor code over with a LT code.

Theorem 3.

Consider a Raptor code over with a LT code, with an outer code characterized by a composition enumerator and an inner LT code with output degree distribution . The probability of decoding failure under ML erasure decoding given that output symbols have been collected by the receiver can be upper bounded as

where is the set of all possible compositions for vectors in .

Finally, the fourth theorem establishes a bound on the probability of decoding failure of a multi-edge Raptor code over with a binary LT code.

Theorem 4.

Consider a multi-edge type Raptor code over with an outer code characterized by a bivariate composition enumerator , and a LT code with bivariate output degree distribution . The probability of decoding failure under ML erasure decoding given that output symbols have been collected by the receiver can be upper bounded as

(9)
(10)

where and are the set of all possible compositions for vectors in and in , respectively.

Theorems 1, 2, 3 and 4 apply to Raptor codes with a given outer code. Next we extend these results to the case of a random outer code drawn from an ensemble of codes. Specifically, we consider a parity-check based ensemble of outer codes, denoted by

, defined by a random matrix of size

whose elements belong to . A linear block code of length belongs to if and only if at least one of the instances of the random matrix is a valid parity-check matrix for it. Moreover, the probability measure of each code in the ensemble is the sum of the probabilities of all instances of the random matrix which are valid parity-check matrices for that code. Note that all codes in are linear, have length , and have dimension . In the following we use the expression Raptor code ensemble to refer to the set of Raptor codes obtained by concatenating an outer code belonging to the ensemble with an LT code. Given a Raptor code ensemble we define its expected probability of decoding failure as

(11)

where the expectation is taken over all codes in the ensemble of outer codes .

The following corollary extends the result of Theorem 1 to Raptor code ensembles.

Corollary 1.

Consider a Raptor code ensemble over with an outer code selected from the ensemble , characterized by an expected weight enumerator and an LT code with degree distribution . Under ML erasure decoding and given that output symbols have been collected by the receiver, the expected probability of the decoding failure can be upper bounded as

The following three corollaries extend Theorems 2, 3, 4 and to Raptor code ensembles.

Corollary 2.

Consider a multi-edge type Raptor code ensemble over , whose outer code is selected from a code ensemble characterized by an expected bivariate weight enumerator polynomial and an inner LT code with bivariate output degree distribution . The expected probability of decoding failure under ML erasure decoding given that output symbols have been collected by the receiver can be upper bounded as

where is defined in (8).

Corollary 3.

Consider an ensemble of Raptor codes over with a LT code, where the outer code is selected from a code ensemble characterized by an expected composition enumerator . The expected probability of decoding failure under ML erasure decoding given that output symbols have been collected by the receiver can be upper bounded as

where is the set of all possible compositions for vectors in .

Corollary 4.

Consider a multi-edge type Raptor code ensemble over with a LT code, where the outer code is selected from an ensemble characterized by an expected bivariate composition enumerator , and the LT code has bivariate output degree distribution . The expected probability of decoding failure under ML erasure decoding given that output symbols have been collected by the receiver can be upper bounded as

(12)
(13)

where and are the set of all possible compositions for vectors in and in , respectively.

V Derivation of the Upper Bounds

This section contains the proofs of the results presented in Section IV.

V-a Proof of Theorem 1

An ML decoder solves the linear system of equations in (4). Decoding fails whenever the system does not admit a unique solution, that is, if and only if , i.e., if . For any two vectors and , we define as the event , and as the event . We have

(14)

where we made use of the fact that due to outer code linearity, the all zero intermediate word is only generated by the all zero input vector.

Due to linearity of the outer code, if , then for any . Furthermore, for any , if and only if . Thus, for any two outer codewords and such that for some , the event holds if and only if does, and we have . If we take a union bound on (14), this allows us dividing it by a factor , leading to

(15)

Defining as the set of codewords of Hamming weight in the outer code ,

the expression can be developed as

(16)
(17)

where we made use of the fact that, since the neighbors of an output symbol are chosen uniformly at random, does not depend on the specific vector , but only on its Hamming weight.

Observing that the output symbols are independent of each other, we have

where .

Let and

be discrete random variables representing the number of intermediate symbols which are linearly combined to generate the generic output symbol

, and the number of non-zero such intermediate symbols, respectively. Note that . An expression for may be obtained as

where is due to

and to

Letting , since the intermediate symbols are chosen uniformly at random by the LT encoder we have

(18)

Let us denote by and let us observe that the non-zero elements of are i.i.d. and uniformly drawn in . On invoking Lemma 1 in Appendix  A,777The proof in Appendix A is only valid for fields with characteristic , the case of most interest for practical purposes. The proof of the general case is a simple extension of Lemma 1. we have

(19)

We conclude that is given by

(20)

where and are given by (18) and (19), respectively.

Expanding this expression and rewriting it using Krawtchouk polynomials and making use of the Chu-Vandermonde identity, one obtains (7).888The expression of was derived in [17], where an upper bound on the performance of LT codes was derived. However, the derivation of in [17] is different from the one we provide in this paper.

We remark that (15) holds not only for Raptor codes over , but also for the other three considered constructions. Hence, (15) represents the starting point in all subsequent proofs.

V-B Proof of Theorem 2

For this construction we may develop (15) as

(21)

where is the set of codewords in with non-zero elements in and non-zero elements in , formally

Making use of the bivariate weight enumerator of the outer code, we can rewrite (21) as

(22)

where we made use of the fact that since the neighbors of an output symbol are chosen uniformly at random, does not depend on the particular vector , but only on its split Hamming weight, and .

Since output symbols are generated independently of each other

where .

Let and be two discrete random variables representing, respectively, the number of intermediate symbols of type which are linearly combined to generate output symbol , and the number of non-zero such intermediate symbols. Similarly, let and be two discrete random variables representing, respectively, the number of intermediate symbols of type which are linearly combined to generate output symbol , and the number of non-zero such intermediate symbols. Note that we have and . The expression of can be obtained as

(23)
(24)
(25)
(26)
(27)
(28)
(29)

where is due to

(30)

is due to

(31)

and follows from independence of and . Let us denote by . Since the non-zero elements of are i.i.d. and uniformly drawn in , on invoking Lemma 1 in the Appendix we have

(32)

Similarly, letting , we have

If we now define and use the same reasoning for the intermediate symbols of type , we have

Hence, the expression of is given by

Finally, after expanding and rewriting this expression using Krawtchouk polynomials, we obtain (8). ∎

V-C Proof of Theorem 3

Starting again from (15) and defining as the set of codewords with composition in the outer code ,

we have

(33)
(34)

where we made use of the fact that since the neighbors of an output symbol are chosen uniformly at random, any two codewords having the same composition are characterized by the same probability .

Due to independence among the output symbols, we have

(35)

Let us now introduce again an auxiliary discrete random variable to represent the output symbol degree, i.e., the number of intermediate symbols which are summed to generate the generic output symbol . We have

(36)

Next, let us introduce the random vector representing the composition of the intermediate output symbols that are added to obtain output symbol . Recalling that is the set of possible compositions of length- vectors, we can recast as

(37)
(38)
(39)

where the indicator function has been defined in Section II. The term

can easily be computed making use of a multivariate hypergeometric distribution. In particular, we have

(40)

V-D Proof of Theorem 4

The proof tightly follows the proofs of Theorems 2 and 3. Let us define as the set of codewords in where have, respectively, composition and , formally

From (15) we obtain

(41)
(42)

Again we exploited the fact that since the neighbors of an output symbol are chosen uniformly at random,