An Entropy-based Proof of Threshold Saturation for Nonbinary SC-LDPC Ensembles on the BEC

by   Zhonghao Zhang, et al.

In this paper we are concerned with the asymptotic analysis of nonbinary spatially-coupled low-density parity-check (SC-LDPC) ensembles defined over GL(2^m) (the general linear group of degree m over GF(2)). Our purpose is to prove threshold saturation when the transmission takes place on the binary erasure channel (BEC). To this end, we establish the duality rule for entropy for nonbinary variable-node (VN) and check-node (CN) convolutional operators to accommodate the nonbinary density evolution (DE) analysis. Based on this, we construct the explicit forms of the potential functions for uncoupled and coupled DE recursions. In addition, we show that these functions exhibit similar monotonicity properties as those for binary LDPC and SC-LDPC ensembles over general binary memoryless symmetric (BMS) channels. This leads to the threshold saturation theorem and its converse for nonbinary SC-LDPC ensembles on the BEC, following the proof technique developed by S. Kumar et al.


page 1

page 2

page 3

page 4


Spatially Coupled Generalized LDPC Codes: Asymptotic Analysis and Finite Length Scaling

Generalized low-density parity-check (GLDPC) codes are a class of LDPC c...

Improving the Decoding Threshold of Tailbiting Spatially Coupled LDPC Codes by Energy Shaping

We show how the iterative decoding threshold of tailbiting spatially cou...

EBP-GEXIT Charts for M-ary AWGN Channel for Generalized LDPC and Turbo Codes

This work proposes a tractable estimation of the maximum a posteriori (M...

On Doped SC-LDPC Codes for Streaming

In streaming applications, doping improves the performance of spatially-...

Spatially coupled turbo-like codes: a new trade-off between waterfall and error floor

Spatially coupled turbo-like codes (SC-TCs) have been shown to have exce...

Connecting Spatially Coupled LDPC Code Chains for Bit-Interleaved Coded Modulation

This paper investigates the design of spatially coupled low-density pari...

Analysis of peeling decoder for MET ensembles

The peeling decoder introduced by Luby, et al. allows analysis of LDPC d...

I Introduction

Spatial coupling has been recognized as an effective way of improving the performance of low-density parity-check (LDPC) codes. This concept was first introduced in [5695130], but its underlying idea can be traced back to the benchmark work by Zigangirov [782171] for the design of LDPC codes with convolutional structures. The resultant codes, termed as spatially-coupled LDPC (SC-LDPC) codes, are found to have better error correction capability than the uncoupled ones in terms of decoding threshold [1362891][5571910]. This finding motivates the applications of the underlying principle behind SC-LDPC codes to a wide variety of communication systems with much success. See [6655077][6990513][7084636] for coded modulation systems, [6034031][6364433][6387167] for inter-symbol interference channels, and [6034088][6120387][7031889] for multiple access channels.

From a design point of view, it is of particular importance to predict the asymptotic performance gain introduced by SC-LDPC codes compared with standard LDPC block codes. This can be done by calculating the belief propagation (BP) threshold of SC-LDPC codes based on the coupled DE algorithm, but at a cost of high complexity. A more efficient way is to prove the existence of the threshold saturation effect. For example, for a binary regular SC-LDPC code, it has been shown that the BP threshold saturates to the maximum-a-posteriori (MAP) threshold of its underlying uncoupled LDPC codes on the BEC [5695130] and general binary memoryless symmetric (BMS) channels [6589171]. As the MAP threshold of the binary regular LDPC code can be (tightly) calculated based on the generalized extrinsic information transfer (GEXIT) chart [Richardson:2008:MCT:1795974], this theoretic result provides a simple guidance to predetermine the asymptotic BP threshold of a regular SC-LDPC code, avoiding the need of the coupled DE algorithm.

For general SC-LDPC coded systems characterized by scalar recursions (e.g., the DE recursion for a binary irregular SC-LDPC code on the BEC), Yedla et al. introduced a technique based on potential functions for the proof of threshold saturation [6325197][6887298]. The underlying idea behind this technique is to construct real-valued potential functions by taking an integral of scalar DE recursions (e.g., the areas under the transfer curves in the EXIT chart). By doing this, Yedla et al. proceeded the analysis of the DE fixed points by investigating the stationary points of the potential functions. They proved that, the threshold of a scalar coupled DE recursion asymptotically coincides with the potential threshold of the uncoupled DE recursion defined by the vanishing of the so-called energy gap (a local minimum of the underlying potential function). This technique can be directly applied to binary irregular SC-LDPC codes on the BEC, proving the existence of the threshold saturation effect in this scenario.

The work by [6325197][6887298] was proposed for scalar recursions. The main difficulty of its extension to nonscalar recursions is how to construct potential functions by taking an integral over the space of density functions. For SC-LDPC codes on the BMS channels, S. Kumar et al circumvented this difficulty by specializing the replica-symmetric (RS) free entropy functional to LDPC ensembles and derived the potential functions based on entropies [6912949]. It turns out that these functions are the negative of the RS free entropies associated with the code ensembles. Their analysis shed a light on the invaluable role of the duality rule for entropy [Richardson:2008:MCT:1795974] in the construction of potential functions. This rule reveals an entropy conservation relation involving the variable-node (VN) and the check-node (CN) convolutional operators [measson2006conservation], establishing the bridge between the DE fixed points and the stationary points of potential functions. Following the idea by S. Kumar et al., we are able to extend the entropy-based proof technique to binary irregular SC-LDPC ensembles on general BMS channels.

The performance gain introduced by employing nonbinary SC-LDPC codes has been numerically observed [5706907][6874959][7024893]. It arises a natural question whether the threshold saturation effect also exists in such scenarios. Motivated by this, the authors in [7430313] studied nonbinary SC-LDPC codes defined over the general linear group when the transmission takes place on the BEC. They concluded that, to apply the proof technique by Yedla et al

, one should first identify the existence of the potential functions for the nonscalar DE recursions. For this reason, the authors developed a constructive criterion that is applicable to general vector spatially-coupled recursions defined over general multivariate polynomials. Although the authors conjectured that potential functions always exist, it seems not an easy task to construct these functions except for some special cases (see Table II therein).

In this paper, we focus on the asymptotic performance of nonbinary SC-LDPC ensembles defined over the general linear group GL and prove that the threshold saturation effect indeed occurs for transmission on the BEC. Our work is a nonstraightforward extension of [7430313] and [6912949]. Our contribution is three-fold.

  • First of all, we establish the duality rule for entropy for nonbinary DE recursions on the BEC. As in the binary case mentioned above, this rule also reveals a conservation relation between the input and the output entropy of nonbinary VN and CN convolutional operators and is the key step towards constructing potential functions in the proof of threshold saturation.

  • Secondly, we propose the explicit forms of nonbinary potential functions similar to those in [6912949] derived for binary SC-LDPC ensembles over BMS channels. This proves the conjecture proposed in [7430313] for all code degree distributions and . We further show that these potential functions exhibit similar monotonicity properties including the partial order preservation properties. This finding implies that it is possible to develop the threshold saturation theorem and its converse for nonbinary SC-LDPC ensembles on the BEC, following the idea by S. Kumar et al [6912949].

  • Finally, we modify the definition of the energy gap that is used to calculate the potential threshold of the underlying LDPC ensemble. In specific, the energy gap in [6912949] is defined based on the infimum over the complementary subset of the basin of attraction to the trivial DE fixed point, while in our work we restrict the complementary subset to the set of nontrivial underlying DE fixed points (see Definition 11).

The remainder of the paper is organized as follows. In Section II, we define nonbinary LDPC and SC-LDPC ensembles concerned in this paper and briefly discuss the form of the density in the nonbinary DE analysis. In Section III, we review the definitions of the entropy function and the VN and CN convolutional operators. We establish and prove several important identities and properties including the duality rule for entropy and the partial order preservation properties. In Section IV, we construct potential functions for nonbinary uncoupled and coupled DE recursions. The monotonicity properties of these functions are also proposed and proved based on the theoretic results in Section III. We establish the threshold saturation theorem and its converse at the end of Section IV. Finally, Section V concludes the whole paper.

I-a Notations

We use to represent the set of all real numbers and define and . For any , we define . The two integers and denote the coupling length and the coupling width for an SC-LDPC ensemble, respectively. By defining , we introduce and to denote the positions of VNs and CNs, respectively. Further, define where represents the maximum integer less than or equal to .

Ii Preliminaries

Ii-a LDPC and SC-LDPC Ensembles Defined Over GL

Denote by LDPC the nonbinary LDPC ensemble defined over the general linear group GL. Here we omit the codeword length for notational brevity, since in this paper we always restrict ourselves to the limit where the codeword length trends to infinity. Following the standard notational convention, we use and to denote the edge-perspective degree distributions of VNs and CNs, respectively, with nonnegative coefficients and satisfying . We also adopt node-perspective degree distributions denoted as and , the coefficients of which are determined by [Richardson:2008:MCT:1795974]


A nonbinary LDPC code selected from LDPC can be described in the form of a bipartite graph termed the Tanner graph. Each VN in the Tanner graph corresponds to a coded symbol defined over GF. When the transmission takes place on the BEC, it is convenient to write the coded symbol in the form of a binary column vector of bits, i.e. with . With this notation, we can represent the coding constraint imposed by each CN as follows


where 0 denotes the zero vector of length , the subset of VNs connected to CN , and , a binary -by-invertible matrix uniformly selected from GL at random, is the label of the edge from VN to CN in the Tanner graph.

We also consider the nonbinary SC-LDPC ensemble over GL denoted as SC-LDPC in this paper. Such an ensemble can be constructed from the graphic perspective as follows. As illustrated in Fig. 1, we first place the Tanner graphs of LDPC along a chain, the positions of which are indexed by an integer . Next, at each position , the outgoing edges of the VNs are uniformly and randomly divided into groups, being reconnected to those CNs at positions . Likewise, the CNs at each position are also uniformly and randomly connected to the VNs at positions . After that, we terminate the coupling chain by removing the VNs at positions and their outgoing edges. As a result, all CNs with degree less than two become invalid and thus are also removed from the coupling chain. The resultant graph is referred to as the Tanner graph of SC-LDPC. The termination procedure will reduce the degrees of some CNs at the two ends of the coupling chain. A coding rate loss is introduced, but it will vanish as (while keeping fixed). More importantly, the termination procedure leads to a phenomenon termed decoding wave propagation in the BP decoding algorithm, which is the fundamental mechanism behind threshold saturation.

One may equivalently define SC-LDPC from the parity-check matrix perspective. See [7430313] for details.

Fig. 1: The Tanner graph for SC-LDPC, where each square (resp., circle) represents a collection of multiple CNs (resp., VNs) of the underlying LDPC ensemble located at that position. The dashed squares, circles and edges are removed in the termination procedure.

Ii-B Densities of Messages in BP Decoding

In the DE analysis, we are interested in tracking the distributions of messages exchanged in the BP decoding algorithm. These distributions are referred to as the densities. In general, density tracking is difficult for nonbinary LDPC ensembles since the decoding performance may depend on the transmitted codeword with possible values for each coded symbol . Fortunately, in the case where the transmission takes place on the BEC and the edge labels are defined over GL, the form of the density can be simplified. First of all, thanks to the symmetry of the BEC, the BP decoding performance does not depend on the specific transmitted codeword, therefore we can assume that the all-zero codeword is transmitted [Richardson:2008:MCT:1795974]

. Under this assumption, the a posteriori probability mass function (PMF) of

is equiprobable over a subspace of the -dimensional binary vector space [1561993]. Consider an example where , and with being the channel observation containing erased bits “”. In this example, the a posteriori PMF is given by if takes values from the subspace of dimension , and otherwise. Secondly, it can be shown that the subspace dimension does not change when a message is passed along an edge in the BP decoding algorithm. To see this, notice that if the a posteriori PMF of is equiprobable over , then the a posteriori PMF of is equiprobable over . Obviously, the dimensions of and are identical due to the fact that the binary matrix is invertible. Therefore, it is sufficient to keep track the subspace dimensions instead of the a posteriori PMFs of coded symbols [1561993].

For the above reason, in this paper, our discussions are based on the density with the following form as in [7430313][1561993].

Definition 1

The density of a message in the BP decoding algorithm for LDPC and SC-LDPC on the BEC is defined as the probability vector of length , the -th entry of which is the probability that the a posteriori PMF corresponding to the message is equiprobable over a subspace of dimension . In what follows, the set of all such densities will be denoted as , i.e.,


For notational brevity, we will also use to represent the -th entry of , ,

There are two extremal densities in , one of which is corresponding to the case where the message offers no information about the coded symbol, and the other is corresponding to the error-free case where the coded symbol can be recovered from the message perfectly. Further, we will use to denote the density with the -th entry being and others being , .

Iii Duality Rule for Entropy and
Partial Ordering

Iii-a The Duality Rule for Entropy

In this subsection, we will establish the duality rule for entropy for nonbinary LDPC and SC-LDPC ensembles on the BEC. To this end, we first present and review the definitions of the entropy function and the basic VN and CN operators.

Definition 2

For any , the entropy function of is defined as

Remark 1

The entropy function can be regarded as a measure of the average uncertainty of a message, the distribution of which can be determined by . As discussed in Subsection II-B, the a posteriori PMF of a message is always equiprobable over a subspace . Let be the dimension of . Since there are elements in with equal probability, the uncertainty of this message is bits. Therefore, if we treat

as a random variable with

being the distribution, then the average uncertainty of the message is given by bits.

In this paper, we adopt the notions and introduced in [1561993] for the VN and CN convolutional operators.

Definition 3

For any , and are two densities, the -th entries of which are respectively given by


. Here, the coefficients and are respectively given by


with being the Gaussian binomial coefficient defined as follows


In addition, we define and for , and we use the convention that and if .

For notational convenience, in the sequel, we will use to denote either or . In Appendix C, we will prove the commutative, distributive and associative laws of and apply them to the derivative analysis of the entropy function.

Remark 2

In the sequel, we will compute the difference between the entropies of two densities involving the convolutional operator , e.g., . For notational convenience, we will extend Definitions 2 and 3 to all real-valued vectors of length (not necessarily the probability vectors). By doing this, we can rewrite as .

We are now ready for the duality rule for entropy for nonbinary LDPC and SC-LDPC ensembles on the BEC.

Lemma 1

For any ,


By Definition 2,


Therefore, it is suffice to show that, for any ,


Although (10) can be verified for small values of , how to prove it for all is the most difficult step in the proof of Lemma 1. One may consider the method by induction on . However, such an idea is perhaps not feasible since the relation between and is quite involved. To circumvent this difficulty, we construct two bivariate functions (see (17) and (26) below), and by taking their partial derivatives we will obtain two polynomials whose coefficients are related to the left-hand side of (10). This will lead to the desired result.

Now we proceed the proof of Lemma 1 with the following Gauss’s binomial formula [kac2001quantum],


For , define as follows


By applying (11) to (12), we obtain


where (a) is obtained by replacing and with and , respectively.

Now we take the partial derivative of in (12) with respect to , then multiply the result by , and finally replace with . This leads to the following result


Applying the same procedure to (15) yields


Putting the above together, we obtain


Now we consider the following identity deduced from (11) by replacing with ,


Similarly, for , define as follows


Again, applying (19) to (20) yields


where (a) is obtained by replacing and with and , respectively.

Following the same procedure as in (16)-(18), we can show that


Now, by substituting and (see (108) in Appendix B) into (26) and replacing with , we can deduce that


Next, by rearranging and combining (18) and (27) as follows and substituting (11) to the term , we have




Finally, by substituting (30) to (9), we complete the proof of Lemma 1.

Remark 3

Now let us briefly discuss (30) and interpret the operational meaning of the duality rule for entropy (8). For any fixed , consider two statistically independent messages, the a posteriori PMFs of which are equiprobable over a subspace of dimension and a subspace of dimension , respectively. As discussed in Remark 1, the uncertainties of the two messages are given by bits and bits, and therefore, the total uncertainty is given by bits. If we combine the two messages based on the VN (resp. CN) decoding algorithm, then the a posteriori PMF of the combined message is equiprobable over the intersection (resp. sum) of and , denoted as (resp. ). Moreover, as interpreted in Subsection II-A in [7430313], (resp. ) is the probability of the event that the dimension of (resp. ), or equivalently, the uncertainty of the combined message, is exactly (bits). Therefore, the identity (30) indicates that the total (average) uncertainty is invariant under the combinations of two statistically independent messages based on the VN and the CN decoding algorithms. This explains why we mentioned in the introduction that the duality rule for entropy (8) reveals a conservation relation between the input and the output entropies of VNs and CNs.

Following the same line as in [6912949], we extend the rule (8) to the following relations and omit the details for brevity.

Corollary 1

For any ,


Iii-B Partial Ordering

An important issue in the DE analysis is comparing two densities to identify which one offers more information about the coded symbols. For this purpose, a concept termed partial ordering is established in [Richardson:2008:MCT:1795974] based on statistical degradation in the context of binary LDPC ensembles over BMS channels. In [7430313]

, the authors defined partial ordering based on the complementary cumulative distribution function to accommodate the analysis of nonbinary LDPC ensembles on the BEC. We will exploit the notion of partial ordering in

[7430313] in this paper, the definition of which is reformulated as follows.

Definition 4

For any , we say that or if the following inequality holds


and say that or if and .

Proposition 1

For any , the strict partial order holds if and only if there exists a nonempty set such that for and for .


The proof is straightforward and we omit it for simplicity.

It is easy to justify that .

Proposition 2

Consider a series of densities . The limit exists if either or holds for all .


By definition, we can deduce that either or holds for each . Notice that is always bounded between and . Therefore, the limit does indeed exist.

Proposition 3

The entropy function preserves partial ordering. More precisely, for any , we have if , and if .


By Definition 4, implies that . Therefore


where (a) is based on Proposition 4 in Appendix A.

The proof of the implication now becomes straightforward by Proposition 1.

Lemma 2

The VN and CN convolutional operators and preserve partial ordering. More precisely, for any with , we have


Further, if , then


We focus on the results for . The proof for is identical by noticing that .

Notice that, for , implies by Definition 4. Therefore


where (a) and (c) are based on the fact that if , (b) follows from Proposition 4 in Appendix A, and (d) is based on Claim 3) of Proposition 5 in Appendix B. Therefore, by Definition 4, we obtain the desired result .

Next, we show that preserves strict partial ordering. To this end, we rearrange the above as follows


Let be two integers satisfying and . The existence of such and is guaranteed by the assumption and . By letting and discarding some nonpositive terms in (42), we can obtain a strictly negative upper bound, i.e.,


Thus holds for at least one integer . This completes the proof of (36).

Remark 4

The partial order preservation property in Lemma 2 guarantees that, if an uncoupled DE recursion (see (61) in the sequel) is initialized by , i.e., the most “uncertain” density, then the densities generated by the DE recursion are always partially ordered as the iteration proceeds.

Lemma 3

For any with and , we have


where the equalities hold if and only if and .


For any , we define


Obviously, if and . Further, if , then


On the other hand, based on (18), for , we have


where (a) is based on (11) and (b) is obtained by and . Therefore, we have


Following (46), we have


By Definition 4, , and imply and , respectively. The first inequality in (44) can be deduced from (51). Specifically,


Further, due to the strict positiveness of the right-hand side of (51), the equality in (55) holds if and only if and .

Now, based on (