On The Reliability Function of Discrete Memoryless Multiple-Access Channel with Feedback

01/23/2018 ∙ by Mohsen Heidari, et al. ∙ University of Michigan 0

We derive a lower and upper bound on the reliability function of discrete memoryless multiple-access channel (MAC) with noiseless feedback and variable-length codes (VLCs). The bounds are similar to the one for point-to-point channels, increasing linearly with respect to an appropriate distance between the rate pair and the capacity boundary. For the lower bound on the error exponent, we adapt Yamamoto and Itoh's coding scheme consisting of a data and a confirmation stage. In the data stage we use arbitrary feedback capacity-achieving codes. In the confirmation stage, each transmitter sends one bit of information to the receiver using a pair of codebooks of size two, one for each transmitter. The codewords at this stage are selected randomly according to an appropriately optimized joint probability distribution. For the upper bound on the error exponent, we adopt the proof techniques of Burnashev for the reliability function of the point-to-point case. The upper bound is derived by studying the rate of decrease of appropriate message entropies.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Noiseless feedback does not increase the capacity for communications over discrete memoryless channels (DMC) [1]. Furthermore, Dobrushin [4] and later Haroutunian [5] showed that feedback does not improve the error exponent of symmetric channels when fixed-length codes are used. Nevertheless, feedback can be very useful in the context of variable-length codes.

In a remarkable work, Burnashev [2] demonstrated that the error exponent improves for DMCs with feedback and variable-length codes. The error exponent has a simple form


where is the (average) rate of transmission, is the capacity of the channel, and is the maximal relative entropy between conditional output distributions. Berlin et al [6] have provided a simpler derivation of the Burnashev bound that emphasizes the link between the constant and the binary hypothesis testing problem. Yamamoto and Itoh [7] introduced a coding scheme that its error exponent achieves in (1). Their scheme consists of two distinct transmission phases that we called the data and the confirmation phase, respectively. In the data stage the message is encoded using a capacity achieving fixed blocklength code. During the confirmation phase, the transmitter sends one bit of information to the receiver. The decoder performs a binary hypothesis test to decide if or is transmitted.

In the context of communications over multi-user channels, the benefits of feedback are more prominent. For instance, Gaarder and Wolf [8] showed that feedback can expand the capacity region of discrete memoryless multiple-access channels (MAC). Willems [9] derived the feedback-capacity region for a class of MACs. Characterizing the capacity region and the error exponent for general MACs remains an open problem. Using directed information measures, Kramer [10] was able to characterize the feedback-capacity region of two-user MAC with feedback. However, the characterization is in the form of infinite letter directed information measures which is not computable in general. The error exponent for discrete memoryless MAC without feedback is studied in [13, 14].

In this paper, we study the error exponent of discrete memoryless MAC with noiseless feedback. In particular, we derive an upper-bound and a lower-bound. For that, let denote the polar coordinate of in . In this setting, the upper-bound is


where is the point of the capacity frontier at the angle determined by . The lower-bound is the same as but with different constant . The constants and are determined by the relative entropy between the conditional output distributions. We show that for a class of MACs the two bounds coincide.

The paper is organized as follows: In Section II, basic definitions and the problem formulation are provided. In Section III, we derive a lower-bound for the reliability function. In Section IV, we characterize an upper-bound for the reliability function. In Section V, we compare the lower and upper-bound and explore examples for the tightness of the bounds. Finally, Section VI concludes the paper.

Ii Problem Formulation and Definitions

Consider a discrete memoryless MAC with input alphabets , and output alphabet

. The channel conditional probability distribution is denoted by

for all . Such setup is denoted by . Let and , be the channel output and the inputs sequences after uses of the channel, respectively. Then, the following condition is satisfied:


We assume that the output of the channel as a feedback is available at the encoders with one unit of delay.

Definition 1.

An - variable-length code (VLC) for a MAC with feedback is defined by

  • A pair of messages

    selected randomly with uniform distribution from


  • Two sequences of encoding functions

    one for each transmitter.

  • A sequence of decoding functions

  • A stopping time with respect to (w.r.t) the filtration defined as the -algebra of for . Furthermore, it is assumed that satisfies .

For each , given a message , the th output of Transmitter is denoted by .

Let . Then, the decoded messages at the decoder are denoted by , and . In what follows, for any VLC, we define average rate-pair, error probability, and error exponent. Average rates for an VLC are defined as

The probability of error is defined as

The error exponent of a VLC with probability of error and stopping time is defined as .

Definition 2.

A reliability function is said to be achievable for a given MAC, if for any and there exists an -VLC such that

where , and is the error probability of the VLC.

Definition 3.

The reliability function of a MAC with feedback is defined as the supremum of all achievable reliability functions .

Ii-a The Feedback-Capacity Region of MAC

We summarize Kramer’s results presented in [10] for the feedback capacity of MAC. We use directed information and conditional directed information as defined in [10]. The normalized directed information from a sequence to a sequence when causally conditioned on is denoted by


The feedback-capacity region of a discrete memoryless MAC with feedback is denoted by , and is the closure of the set of all rate-pairs such that

where is a positive integer, and factors as

Definition 4.

Let , and . Define

where factors as in (5).

Fact 1.

The feedback-capacity of a discrete memoryless MAC with feedback is the same as the closure of the set of rate-pairs such that the inequality

holds for all , with .

Ii-B Notational Conventions

For more convenience, we denote a rate-pair by , where . For a MAC we use the following notational convenience


The Kullback–Leibler divergence for the MAC with transition probability matrix

is defined as

where . For notational convenience we denote

Iii A Lower-Bound for the Reliability Function

We build upon Yamamoto-Itoh transmission scheme for point-to-point (ptp) channel coding with feedback [7]. The scheme sends the messages through blocks of length . The transmission process is performed in two stages: 1) The “data transmission” stage taking up to channel uses, 2) The “confirmation” stage taking up to channel uses, where is a design parameter taking values from .

Stage 1

For the first stage, we use any coding scheme that achieves the feedback-capacity of the MAC. The length of this coding scheme is at most . Let

denote the decoder’s estimation of the messages at the end of the first stage. Define the following random variables:

Because of the feedback, and are known at each transmitter. Therefore, at the end of the first stage, transmitter has access to , and , where .

Stage 2

The objective of the second stage is to inform the receiver whether the hypothesis or is correct. For that, each transmitter employs a code of size two and length . The codewords of such codebooks are denoted by two pairs of sequences and each with elements belonging to . Fix a joint-type defined over the set and for sequences of length . The sequences are selected randomly among all the sequences with joint-type . During this stage and given , Transmitter sends . Similarly, Transmitter 2 sends .


Upon receiving the channel output, the receiver estimates . Denote this estimation by . If , then the hypothesis is declared. Otherwise, is declared. Because of the feedback, is also available at each encoders. If , then transmission stops and a new data packet is transmitted at the next block. Otherwise, the message is transmitted again at the next block. The process continues until occurs.

The confirmation stage in the proposed scheme can be viewed as a decentralized binary hypothesis problem in which a binary hypothesis is observed partially by two distributed agents and the objective is to convey the true hypothesis to a central receiver. This problem is qualitatively different from the sequential binary hypothesis testing problem as identified in [6] for ptp channel. Note also that in the confirmation stage we use a different coding strategy than the one used in Yamamoto-Itoh scheme [7]. Here, all four codewords have a joint-type . It can be shown that repetition codes, and more generally, constant composition codes are strictly suboptimal in this problem.

Theorem 1.

The following is a lower-bound for the reliability function of any discrete memoryless MAC:




and the supremum is taken over all probability distributions defined over .


The proof is given in Appendix A. ∎

Iv An Upper-bound for the Reliability Function

In this part of the paper, we establish an upper-bound for the reliability function of any discrete memoryless MAC. Define

Theorem 2 (Upper-bound).

For any VLC with probability of error , and any , there exists a function such that the following is an upper-bound for the reliability function of the VLC


where is the rate pair of the VLC and satisfies

Corollary 1.

From Theorem 2, the following is an upper-bound for the error exponent of a MAC:

where , and is as in Theorem 2.


The proof is given in Appendix E. ∎

Iv-a Proof of the Upper-Bound

Consider any VLC with probability of error , and stopping time . Suppose the message at Encoder 2, , is made available to all terminals. For the new setup, as is available at the Decoder, the average probability of error is . Note that . We refer to such setup as -assisted MAC. For a maximum a posteriori decoder, after uses of the channel and assuming the realization and , define

where is a fixed real number. Also, let . Note that is a stopping time w.r.t the filtration . The following lemma provides a lower-bound on the probability of error for such setup.

Lemma 1.

The probability of error, , for a hypothesis testing over a -assisted MAC and variable length codes satisfies the following inequality

where are the two hypothesizes and is the stopping time of the variable length code.

Lemma 2.

For a given MAC with finite the following holds

where .

The above lemmas are extensions of Lemma 1 and Proposition 2 in [6] for MAC. The proofs follow from similar arguments and are omitted.

Lemma 3.

Given a MAC with , and for any VLC with probability of error the following holds


where .


Suppose the VLC is used for a -assisted MAC. As discussed before, . We modify the encoding and the decoding functions of the VLC used for the MAC. Let be a subset of the message set . The subset is to be determined at time . The new decoding function, at time , decides whether the message belongs to . The new encoding functions are the same as the original one until the time . Then, after , the transmitters perform a VLC to resolve the binary hypothesis and . This hypothesis problem is performed from to . With these modifications, the error probability of this binary hypothesis problem is a lower-bound on . In what follows, we present a construction for . Then, we apply Lemma 1 to complete the proof.

Let The quantity can be calculated at all terminals. By definition, at time , the inequality holds almost surely for all . This implies that . Hence, by Lemma 2 at time the inequality holds almost surely. We consider two cases and , where is the constant used in the definition of . For the first case, is the set consisting of the message with the highest a posteriori probability. Since , then . In addition, as , then . For the second case, set to be a set of messages such that and . Such set exists, since holds for all messages .

Note that by the above construction, for each case, . Thus, from Lemma 1 and the argument above, the inequality

holds almost surely. Next, we take the expectation of the above expression. The lemma follows by the convexity of and Jensen’s inequality.

Next, we apply the same argument for the case where is available at all the terminals. For that define

and let . By symmetry, Lemma 3 holds for this case and we obtain


Next, define the following stopping times:

Also, let . using a similar argument as in the above, we can show that


For that, after time , we formulate a binary hypothesis problem in which the transmitters determine whether or not. Here, is a subset which is constructed using a similar method as for in the proof of Lemma 3. We further allow the transmitters to communicate with each other after . The maximum of the right-hand sides of (13), (14) and (15) gives a lower-bound on . The lower-bound depends on the expectation of the stopping times . In what follows, we provide a lower-bound on . Define the following random processes.

Lemma 4.

Given a -VLC, for any there exist and a probability distribution that factors as in (5) such that the following inequalities hold almost surely for

where , and is defined as in (6)-(8).


The proof is provided in Appendix B. ∎

We need the following lemma to proceed. The lemma is a result of Lemma 4 in [2], and we omit its proof.

Lemma 5.

For any and , the following inequality holds almost surely w.r.t

From Lemma 4 and the fact that , the processes are submartingales for . In addition, from Lemma 5 and the inequalities , we can apply Doob’s Optional Stopping Theorem for each submartingale . Then, we get:


where .

Lemma 6.

The following inequality holds for each


We prove the lemma for the case . The proof for follows from a similar argument. For , we obtain


Note that the event implies that , and for all . Hence, this event is included in the event . Thus, applying Markov inequality gives

As a result of the above argument, the right-hand side of (17) does not exceed the following

From Fano’s inequality we obtain

The proof is complete from the above inequality. ∎

As a result of the above lemma and (16), the inequality holds. Finally, combining this inequality with (13)-(15) completes the proof of the theorem.

Iv-B An Alternative Proof for the Upper-Bound

In this part of the paper, we provide a series of Lemmas that are used to prove the Theorem. Define the following random processes.

Lemma 7.

For an -VLC with probability of error the following inequality holds


The proof follows from Fano’s Lemma as in [2]. ∎

Lemma 8.

There exists such that, if , then

holds almost surely, where are defined in (11).


The proof is given in Appendix C. ∎

Lemma 9.

For , define random process as


where the function is defined as Then, there exists such that is a submartingale w.r.t .


Suppose for some . Given this event and using the same argument as in the proof of Theorem 1 in [2] we can show that is a submartingale for all . More precisely, the inequality

holds almost surely w.r.t . Taking the expectation of the both sides in the above inequality gives

Thus, is a submartingale for and w.r.t . ∎

Corollary 2.

Suppose are non-negative numbers such that . Define . Then, is a submartingale w.r.t .

The Theorem follows from the above lemma, and the proof is given in Appendix D.

V The Shape of the Lower and Upper Bounds

In this Section, we point out a few remarks on and the lower-bound defined in Theorem 1. Furthermore, we provide an alternative representation for the bounds and show that the lower and upper-bounds match for a class of MACs.

We first compare the lower bound in (9) and the upper-bound in Corollary 1. For a given arbitrary rate pair inside the feedback-capacity of a given MAC, consider a sequence of VLCs with rates and with average probability of error approaching zero. Then, the following holds: