On The Reliability Function of Discrete Memoryless Multiple-Access Channel with Feedback

I Introduction

Noiseless feedback does not increase the capacity for communications over discrete memoryless channels (DMC) [1]. Furthermore, Dobrushin [4] and later Haroutunian [5] showed that feedback does not improve the error exponent of symmetric channels when fixed-length codes are used. Nevertheless, feedback can be very useful in the context of variable-length codes.

In a remarkable work, Burnashev [2] demonstrated that the error exponent improves for DMCs with feedback and variable-length codes. The error exponent has a simple form

E (R) = (1 - \frac{R}{C}) C_{1},

(1)

where $R$ is the (average) rate of transmission, $C$ is the capacity of the channel, and $C_{1}$ is the maximal relative entropy between conditional output distributions. Berlin et al [6] have provided a simpler derivation of the Burnashev bound that emphasizes the link between the constant $C_{1}$ and the binary hypothesis testing problem. Yamamoto and Itoh [7] introduced a coding scheme that its error exponent achieves $E (R)$ in (1). Their scheme consists of two distinct transmission phases that we called the data and the confirmation phase, respectively. In the data stage the message is encoded using a capacity achieving fixed blocklength code. During the confirmation phase, the transmitter sends one bit of information to the receiver. The decoder performs a binary hypothesis test to decide if $0$ or $1$ is transmitted.

In the context of communications over multi-user channels, the benefits of feedback are more prominent. For instance, Gaarder and Wolf [8] showed that feedback can expand the capacity region of discrete memoryless multiple-access channels (MAC). Willems [9] derived the feedback-capacity region for a class of MACs. Characterizing the capacity region and the error exponent for general MACs remains an open problem. Using directed information measures, Kramer [10] was able to characterize the feedback-capacity region of two-user MAC with feedback. However, the characterization is in the form of infinite letter directed information measures which is not computable in general. The error exponent for discrete memoryless MAC without feedback is studied in [13, 14].

In this paper, we study the error exponent of discrete memoryless MAC with noiseless feedback. In particular, we derive an upper-bound and a lower-bound. For that, let $(| | R - - | |, θ_{R})$ denote the polar coordinate of $(R_{1}, R_{2})$ in $R^{2}$ . In this setting, the upper-bound is

E_{u} (R_{1}, R_{2}) = (1 - \frac{| | R - - | |}{C (θ_{R})}) D_{u}

(2)

where $C (θ_{R})$ is the point of the capacity frontier at the angle determined by $R - -$ . The lower-bound is the same as $E_{u}$ but with different constant $D_{l}$ . The constants $D_{l}$ and $D_{u}$ are determined by the relative entropy between the conditional output distributions. We show that for a class of MACs the two bounds coincide.

The paper is organized as follows: In Section II, basic definitions and the problem formulation are provided. In Section III, we derive a lower-bound for the reliability function. In Section IV, we characterize an upper-bound for the reliability function. In Section V, we compare the lower and upper-bound and explore examples for the tightness of the bounds. Finally, Section VI concludes the paper.

Ii Problem Formulation and Definitions

Consider a discrete memoryless MAC with input alphabets $X_{1}, X_{2}$ , and output alphabet $Y$

. The channel conditional probability distribution is denoted by

Q (y | x_{1}, x_{2})

for all

(y, x_{1}, x_{2}) \in Y \times X_{1} \times X_{2}

. Such setup is denoted by

(X_{1}, X_{2}, Y, Q)

. Let

y^{t}

and

x_{i}^{t}

i = 1, 2,

be the channel output and the inputs sequences after

t

uses of the channel, respectively. Then, the following condition is satisfied:

P (y_{t} | y^{t - 1}, x_{1}^{t - 1}, x_{2}^{t - 1}) = Q (y_{t} | x_{1 t}, x_{2 t}) .

(3)

We assume that the output of the channel as a feedback is available at the encoders with one unit of delay.

Definition 1.

An $(M_{1}, M_{2}, N)$ - variable-length code (VLC) for a MAC $(X_{1}, X_{2}, Y, Q)$ with feedback is defined by

A pair of messages $W_{1}, W_{2}$
selected randomly with uniform distribution from
${1, 2, \dots, M_{i}}, i = 1, 2$ .
Two sequences of encoding functions

$e_{i, t} : {1, 2, \dots, M_{i}} \times Y^{t - 1} \to X_{i}, t \in N, i = 1, 2,$

one for each transmitter.
A sequence of decoding functions

$d_{t} : Y^{t} \to {1, 2, . . ., M_{1}} \times {1, 2, . . ., M_{2}}, t \in N .$
A stopping time $T$ with respect to (w.r.t) the filtration $F_{t}$ defined as the $σ$ -algebra of $Y^{t}$ for $t \in N$ . Furthermore, it is assumed that $T$ satisfies $E [T] \leq N$ .

For each $i = 1, 2$ , given a message $W_{i}$ , the $t$ th output of Transmitter $i$ is denoted by $X_{i, t} = e_{i, t} (W_{i}, Y^{t - 1})$ .

Let $({^W}_{1, t}, {^W}_{2, t}) = d_{t} (Y^{t})$ . Then, the decoded messages at the decoder are denoted by ${^W}_{1} = {^W}_{1, T}$ , and ${^W}_{2} = {^W}_{2, T}$ . In what follows, for any $(M_{1}, M_{2}, N)$ VLC, we define average rate-pair, error probability, and error exponent. Average rates for an $(M_{1}, M_{2}, N)$ VLC are defined as

R_{i} ≜ \frac{{log}_{2} M_{i}}{E [T]}, i = 1, 2.

The probability of error is defined as

P_{e} = P (({^W}_{1},_{2}) \neq (W_{1}, W_{2})) .

The error exponent of a VLC with probability of error $P_{e}$ and stopping time $T$ is defined as $E ≜ - \frac{{log}_{2} P_{e}}{E [T]}$ .

Definition 2.

A reliability function $E (R_{1}, R_{2})$ is said to be achievable for a given MAC, if for any $R_{1}, R_{2} > 0$ and $ϵ > 0$ there exists an $(M_{1}, M_{2}, N)$ -VLC such that

- \frac{{log}_{2} P_{e}}{N} \geq E (R_{1}, R_{2}) - ϵ, and% \frac{{log}_{2} M_{i}}{N} \geq R_{i} - ϵ,

where $i = 1, 2$ , and $P_{e}$ is the error probability of the VLC.

Definition 3.

The reliability function of a MAC with feedback is defined as the supremum of all achievable reliability functions $E (R_{1}, R_{2})$ .

Ii-a The Feedback-Capacity Region of MAC

We summarize Kramer’s results presented in [10] for the feedback capacity of MAC. We use directed information and conditional directed information as defined in [10]. The normalized directed information from a sequence $X^{n}$ to a sequence $Y^{n}$ when causally conditioned on $Z^{n}$ is denoted by

I_{n} (X \to Y | | Z) = \frac{1}{n} I (X^{n} \to Y^{n} | | Z^{n}) .

(4)

The feedback-capacity region of a discrete memoryless MAC with feedback $(X_{1}, X_{2}, Y, Q)$ is denoted by $C$ , and is the closure of the set of all rate-pairs $(R_{1}, R_{2})$ such that

	$R_{1}$	$\leq I_{L} (X_{1} \to Y \| \| X_{2})$
	$R_{2}$	$\leq I_{L} (X_{2} \to Y \| \| X_{1})$
	$R_{1} + R_{2}$	$\leq I_{L} (X_{1} X_{2} \to Y),$

where $L$ is a positive integer, and $P_{X_{1}^{L} X_{2}^{L} Y^{L}}$ factors as

L \prod l = 1 P_{1, l} (x_{1 l} | x_{1}^{l - 1} y^{l - 1}) P_{2, l} (x_{2 l} | x_{2}^{l - 1} y^{l - 1}) Q (y_{l} | x_{1, l} x_{2, l}) .

(5)

Definition 4.

Let $λ_{1}, λ_{2}, λ_{3} \geq 0$ , and $λ_{1} + λ_{2} + λ_{3} = 1$ . Define

	$C_{λ - -} = sup L \in N sup P_{X_{1}^{L} X_{2}^{L} Y^{L}}$	$λ_{1} I_{L} (X_{1} \to Y \| X_{2}) + λ_{2} I_{L} (X_{2} \to Y \| X_{1})$
		$+ λ_{3} I_{L} (X_{1} X_{2} \to Y),$

where $P_{X_{1}^{L} X_{2}^{L} Y^{L}}$ factors as in (5).

Fact 1.

The feedback-capacity of a discrete memoryless MAC with feedback is the same as the closure of the set of rate-pairs $(R_{1}, R_{2})$ such that the inequality

λ_{1} R_{1} + λ_{2} R_{2} + λ_{3} (R_{1} + R_{2}) \leq C_{λ - -}

holds for all $λ_{1}, λ_{2}, λ_{3} \geq 0$ , with $λ_{1} + λ_{2} + λ_{3} = 1$ .

Ii-B Notational Conventions

For more convenience, we denote a rate-pair $(R_{1}, R_{2})$ by $(R_{1}, R_{2}, R_{3})$ , where $R_{3} = R_{1} + R_{2}$ . For a $(X_{1}, X_{2}, Y, Q)$ MAC we use the following notational convenience

$I_{L}^{1}$	$≜ I_{L} (X_{1} \to Y \| \| X_{2}),$	(6)
$I_{L}^{2}$	$≜ I_{L} (X_{2} \to Y \| \| X_{1}),$	(7)
$I_{L}^{3}$	$≜ I_{L} (X_{1} X_{2} \to Y) .$	(8)

The Kullback–Leibler divergence for the MAC with transition probability matrix

Q

is defined as

D_{Q} (x_{1}, x_{2} | | z_{1}, z_{2}) = \sum y \in Y Q (y | x_{1}, x_{2}) {log}_{2} \frac{Q (y | x_{1}, x_{2})}{Q (y | z_{1}, z_{2})},

where $(x_{1}, x_{2}), (z_{1}, z_{2}) \in X_{1} \times X_{2}$ . For notational convenience we denote

	$D_{1} (x_{1}, x_{2} \| \| z_{1}, z_{2})$	$= D_{Q} (x_{1}, x_{2} \| \| z_{1}, x_{2})$
	$D_{2} (x_{1}, x_{2} \| \| z_{1}, z_{2})$	$= D_{Q} (x_{1}, x_{2} \| \| x_{1}, z_{2})$
	$D_{3} (x_{1}, x_{2} \| \| z_{1}, z_{2})$	$= D_{Q} (x_{1}, x_{2} \| \| z_{1}, z_{2}) .$

Iii A Lower-Bound for the Reliability Function

We build upon Yamamoto-Itoh transmission scheme for point-to-point (ptp) channel coding with feedback [7]. The scheme sends the messages $W_{1}, W_{2}$ through blocks of length $n$ . The transmission process is performed in two stages: 1) The “data transmission” stage taking up to $n (1 - γ)$ channel uses, 2) The “confirmation” stage taking up to $n γ$ channel uses, where $γ$ is a design parameter taking values from $[0, 1]$ .

Stage 1

For the first stage, we use any coding scheme that achieves the feedback-capacity of the MAC. The length of this coding scheme is at most $n (1 - γ)$ . Let ${^W}_{1}, {^W}_{2}$

denote the decoder’s estimation of the messages at the end of the first stage. Define the following random variables:

Hi=1{^Wi≠Wi},i=1,2.

Because of the feedback, ${^W}_{1}$ and ${^W}_{2}$ are known at each transmitter. Therefore, at the end of the first stage, transmitter $i$ has access to $W_{i}, {^W}_{1}, {^W}_{2}$ , and $H_{i}$ , where $i = 1, 2$ .

Stage 2

The objective of the second stage is to inform the receiver whether the hypothesis $Θ_{0} : ({^W}_{1}, {^W}_{2}) = (W_{1}, W_{2})$ or $Θ_{1} : ({^W}_{1}, {^W}_{2}) \neq (W_{1}, W_{2})$ is correct. For that, each transmitter employs a code of size two and length $γ n$ . The codewords of such codebooks are denoted by two pairs of sequences $(x_{1} - - - (0), x_{2} - - - (0))$ and $(x_{1} - - - (1), x_{2} - - - (1))$ each with elements belonging to $X_{1} \times X_{2}$ . Fix a joint-type $P_{n}$ defined over the set $X_{1} \times X_{2} \times X_{1} \times X_{2}$ and for sequences of length $γ n$ . The sequences $(x_{1} - - - (0), x_{2} - - - (0), x_{1} - - - (1), x_{2} - - - (1))$ are selected randomly among all the sequences with joint-type $P_{n}$ . During this stage and given $H_{1}$ , Transmitter $1$ sends $x_{1} - - - (H_{1})$ . Similarly, Transmitter 2 sends $x_{2} - - - (H_{2})$ .

Decoding

Upon receiving the channel output, the receiver estimates $H_{1}, H_{2}$ . Denote this estimation by ${^H}_{1}, {^H}_{2}$ . If $({^H}_{1}, {^H}_{2}) = (0, 0)$ , then the hypothesis $^Θ = Θ_{0}$ is declared. Otherwise, $^Θ = Θ_{1}$ is declared. Because of the feedback, $^Θ$ is also available at each encoders. If $^Θ = Θ_{0}$ , then transmission stops and a new data packet is transmitted at the next block. Otherwise, the message is transmitted again at the next block. The process continues until $^Θ = Θ_{0}$ occurs.

The confirmation stage in the proposed scheme can be viewed as a decentralized binary hypothesis problem in which a binary hypothesis ${Θ_{0}, Θ_{1}}$ is observed partially by two distributed agents and the objective is to convey the true hypothesis to a central receiver. This problem is qualitatively different from the sequential binary hypothesis testing problem as identified in [6] for ptp channel. Note also that in the confirmation stage we use a different coding strategy than the one used in Yamamoto-Itoh scheme [7]. Here, all four codewords have a joint-type $P_{n}$ . It can be shown that repetition codes, and more generally, constant composition codes are strictly suboptimal in this problem.

Theorem 1.

The following is a lower-bound for the reliability function of any discrete memoryless MAC:

E_{l} (R_{1}, R_{2}) = min \begin{matrix} λ_{1}, λ_{2}, λ_{3} \geq 0 λ_{1} + λ_{2} + λ_{3} = 1 \end{matrix} D_{l} (1 - \frac{\sum_{i} λ_{i} R_{i}}{C_{λ - -}}),

(9)

where,

D_{l} ≜ sup P_{X_{1} X_{2} Z_{1} Z_{2}} min i = 1, 2, 3 E [D_{i} (X_{1}, X_{2} | | Z_{1}, Z_{2})],

(10)

and the supremum is taken over all probability distributions $P_{X_{1} X_{2} Z_{1} Z_{2}}$ defined over $X_{1} \times X_{2} \times X_{1} \times X_{2}$ .

Proof:

The proof is given in Appendix A. ∎

Iv An Upper-bound for the Reliability Function

In this part of the paper, we establish an upper-bound for the reliability function of any discrete memoryless MAC. Define

D_{i}

≜ max \begin{matrix} x_{1}, z_{1} \in X_{1}, x_{2}, z_{2} \in X_{2} \end{matrix} D_{i} (x_{1}, x_{2} | | z_{1}, z_{2}), i = 1, 2, 3.

(11)

Theorem 2 (Upper-bound).

For any $(N, M_{1}, M_{2})$ VLC with probability of error $P_{e}$ , and any $ϵ > 0$ , there exists a function $δ$ such that the following is an upper-bound for the reliability function of the VLC

	$E (R_{1}, R_{2}) \leq$	$min \begin{matrix} λ_{1}, λ_{2}, λ_{3} \geq 0 λ_{1} + λ_{2} + λ_{3} = 1 \end{matrix} min j \in {1, 2, 3} D_{j} (1 - \frac{λ_{j} R_{j}}{C_{λ}})$
		$+ δ (P_{e}, M_{1} M_{2}, ϵ),$		(12)

where $(R_{1}, R_{2})$ is the rate pair of the VLC and $δ$ satisfies

lim ϵ \to 0 lim P_{e} \to 0 lim M_{1} M_{2} \to \infty δ (P_{e}, M_{1} M_{2}, ϵ) = 0.

Corollary 1.

From Theorem 2, the following is an upper-bound for the error exponent of a MAC:

E_{u} (R_{1}, R_{2}) =

min \begin{matrix} λ_{1}, λ_{2}, λ_{3} \geq 0 λ_{1} + λ_{2} + λ_{3} = 1 \end{matrix} D_{u} (1 - \frac{\sum_{i = 1}^{3} λ_{i} R_{i}}{C_{λ}}) + δ,

where $D_{u} = max {D_{1}, D_{2}, D_{3}}$ , and $δ$ is as in Theorem 2.

Proof:

The proof is given in Appendix E. ∎

Iv-a Proof of the Upper-Bound

Consider any $(N, M_{1}, M_{2})$ VLC with probability of error $P_{e}$ , and stopping time $T$ . Suppose the message at Encoder 2, $W_{2}$ , is made available to all terminals. For the new setup, as $W_{2}$ is available at the Decoder, the average probability of error is $P1e≜P{^W1≠W1}$ . Note that $P_{e} \geq P_{e}^{1}$ . We refer to such setup as $W_{2}$ -assisted MAC. For a maximum a posteriori decoder, after $n$ uses of the channel and assuming the realization $Y^{n} = y^{n}$ and $W_{2} = w_{2}$ , define

T_{1}^{δ} ≜ inf {n : max 1 \leq i \leq M_{1} P (W_{1} = i | y^{n}, w_{2}) \geq 1 - δ},

where $δ > 0$ is a fixed real number. Also, let $τ_{1} ≜ min {T, T_{1}^{δ}}$ . Note that $τ_{1}$ is a stopping time w.r.t the filtration ${F_{W_{2}} \times F_{t}}_{t > 0}$ . The following lemma provides a lower-bound on the probability of error for such setup.

Lemma 1.

The probability of error, $P_{e}$ , for a hypothesis testing over a $W_{2}$ -assisted MAC and variable length codes satisfies the following inequality

P_{e} \geq \frac{min {P (H), P (H^{c})}}{4} e^{- D_{1} E [T]},

where ${H, H^{c}}$ are the two hypothesizes and $T$ is the stopping time of the variable length code.

Lemma 2.

For a given MAC with finite $D_{3}$ the following holds

ζ p (w_{1}, w_{2} | y^{n - 1}) \leq p (w_{1}, w_{2} | y^{n}) \leq \frac{p (w_{1}, w_{2} | y^{n - 1})}{ζ},

where $ζ ≜ {min}_{x_{1}, x_{2}, y} Q (y | x_{1}, x_{2})$ .

The above lemmas are extensions of Lemma 1 and Proposition 2 in [6] for MAC. The proofs follow from similar arguments and are omitted.

Lemma 3.

Given a MAC with $D_{3} < \infty$ , and for any $(N, M_{1}, M_{2})$ VLC with probability of error $P_{e}$ the following holds

P_{e} \geq \frac{ζ δ}{4} e^{- D_{1} E [T - τ_{1}]},

(13)

where $ζ ≜ {min}_{x_{1}, x_{2}, y} Q (y | x_{1}, x_{2})$ .

Proof:

Suppose the VLC is used for a $W_{2}$ -assisted MAC. As discussed before, $P_{e} \geq P_{e}^{1}$ . We modify the encoding and the decoding functions of the VLC used for the MAC. Let $H_{1} \subseteq M_{1}$ be a subset of the message set $M_{1}$ . The subset $H_{1}$ is to be determined at time $τ_{1}$ . The new decoding function, at time $T$ , decides whether the message belongs to $H_{1}$ . The new encoding functions are the same as the original one until the time $τ_{1}$ . Then, after $τ_{1}$ , the transmitters perform a VLC to resolve the binary hypothesis ${W_{1} \in H_{1}}$ and ${W_{1} \notin H_{1}}$ . This hypothesis problem is performed from $τ_{1}$ to $T$ . With these modifications, the error probability of this binary hypothesis problem is a lower-bound on $P_{e}$ . In what follows, we present a construction for $H_{1}$ . Then, we apply Lemma 1 to complete the proof.

Let $P_{e}^{1} (y^{n}, w_{2}) ≜ 1 - {max}_{1 \leq i \leq M_{1}} P (W_{1} = i | y^{n}, w_{2}) .$ The quantity $P_{e}^{1} (y^{τ_{1}}, w_{2})$ can be calculated at all terminals. By definition, at time $τ_{1} - 1$ , the inequality $P (W_{1} = i | Y^{τ_{1} - 1}, W_{2}) < 1 - δ$ holds almost surely for all $i \in [1 : M_{1}]$ . This implies that $P_{e}^{1} (Y^{τ_{1} - 1}, W_{2}) > δ$ . Hence, by Lemma 2 at time $τ_{1}$ the inequality $P_{e}^{1} (Y^{τ_{1}}, W_{2}) \geq ζ δ$ holds almost surely. We consider two cases $P_{e}^{1} (y^{τ_{1}}, w_{2}) \leq δ$ and $P_{e}^{1} (y^{τ_{1}}, w_{2}) > δ$ , where $δ$ is the constant used in the definition of $T_{1}^{δ}$ . For the first case, $H_{1}$ is the set consisting of the message with the highest a posteriori probability. Since $P_{e}^{1} (y^{τ_{1}}, w_{2}) \leq δ$ , then $P (H_{1}) \geq 1 - δ$ . In addition, as $P_{e}^{1} (y^{τ_{1}}, w_{2}) \geq ζ δ$ , then $P (H_{1}^{c}) > ζ δ$ . For the second case, set $H_{1}$ to be a set of messages such that $P (H_{1}) > δ / 2$ and $P (H_{1}) < 1 - δ$ . Such set exists, since $P (W_{1} = i | Y^{τ - 1}, W_{2}) < 1 - δ$ holds for all messages $i \in [1 : M_{1}]$ .

Note that by the above construction, for each case, $P (H_{1}) \in [ζ δ, 1 - ζ δ]$ . Thus, from Lemma 1 and the argument above, the inequality

P{^W1≠W1|Yτ,W2}≥ζδ4e−D1E[T−τ|Yτ,W2]

holds almost surely. Next, we take the expectation of the above expression. The lemma follows by the convexity of $e^{- x}$ and Jensen’s inequality.

∎

Next, we apply the same argument for the case where $W_{1}$ is available at all the terminals. For that define

T_{2}^{δ}

≜ inf {n : max 1 \leq j \leq M_{2} P (W_{2} = j | y^{n}, w_{1}) \geq 1 - δ},

and let $τ_{2} ≜ min {T, T_{2}^{δ}}$ . By symmetry, Lemma 3 holds for this case and we obtain

P_{e} \geq \frac{ζ δ}{4} e^{- D_{2} E [T - τ_{2}]} .

(14)

Next, define the following stopping times:

T_{3}^{δ}

≜ inf {n : max i, j P (W_{1} = i, W_{2} = j | y^{n}) \geq 1 - δ} .

Also, let $τ_{3} = min {T, T_{3}^{δ}}$ . using a similar argument as in the above, we can show that

P_{e} \geq \frac{ζ δ}{4} e^{- D_{3} E [T - τ_{3}]} .

(15)

For that, after time $τ_{3}$ , we formulate a binary hypothesis problem in which the transmitters determine whether $(W_{1}, W_{2}) \in H_{3}$ or not. Here, $H_{3}$ is a subset which is constructed using a similar method as for $H_{1}$ in the proof of Lemma 3. We further allow the transmitters to communicate with each other after $τ_{3}$ . The maximum of the right-hand sides of (13), (14) and (15) gives a lower-bound on $P_{e}$ . The lower-bound depends on the expectation of the stopping times $τ_{i}, i = 1, 2, 3$ . In what follows, we provide a lower-bound on $E [τ_{i}]$ . Define the following random processes.

	$H_{t}^{1}$	$≜ H (W_{1} \| F_{W_{2}} \times F_{t}),$
	$H_{t}^{2}$	$≜ H (W_{2} \| F_{W_{1}} \times F_{t}),$
	$H_{t}^{3}$	$≜ H (W_{1}, W_{2} \| F_{t}),$

Lemma 4.

Given a $(M_{1}, M_{2}, N)$ -VLC, for any $ϵ > 0$ there exist $L$ and a probability distribution $P_{X_{1}^{L} X_{2}^{L} Y^{L}}$ that factors as in (5) such that the following inequalities hold almost surely for $1 \leq t \leq N$

	$E [H_{t + 1}^{1} - H_{t}^{1} \| F_{W_{2}} \times F_{t}]$	$\geq - (I_{L}^{1} + ϵ),$
	$E [H_{t + 1}^{2} - H_{t}^{2} \| F_{W_{1}} \times F_{t}]$	$\geq - (I_{L}^{2} + ϵ),$
	$E [H_{t + 1}^{3} - H_{t}^{3} \| F_{t}]$	$\geq - (I_{L}^{3} + ϵ) .$

where $i = 1, 2, 3$ , and $I_{L}^{i}$ is defined as in (6)-(8).

Proof:

The proof is provided in Appendix B. ∎

We need the following lemma to proceed. The lemma is a result of Lemma 4 in [2], and we omit its proof.

Lemma 5.

For any $t \geq 1$ and $i = 1, 2, 3$ , the following inequality holds almost surely w.r.t $F_{W_{1}} \times F_{W_{2}} \times F_{t}$

log H_{t}^{i} - log H_{t + 1}^{i} \leq max \begin{matrix} j, l \in [1 : M_{1}] k, m \in [1 : M_{2}] \end{matrix} max y \in Y \frac{{^Q}_{j, k} (y)}{{^Q}_{l, m} (y)} .

From Lemma 4 and the fact that $H_{t}^{i} \leq {log}_{2} M_{i} < \infty$ , the processes ${H_{t}^{i} + (I_{L}^{1} + ϵ) t}_{t > 0}$ are submartingales for $i = 1, 2, 3$ . In addition, from Lemma 5 and the inequalities $E [τ_{i}] \leq E [T] \leq N < \infty$ , we can apply Doob’s Optional Stopping Theorem for each submartingale ${H_{t}^{i} + (I_{L}^{1} + ϵ) t}_{t > 0}$ . Then, we get:

log M_{i}

\leq E [H_{τ_{i}}^{i}] + E [τ_{i}] (I_{L}^{i} + ϵ)

(16)

where $M_{3} = M_{1} M_{2}$ .

Lemma 6.

The following inequality holds for each $i = 1, 2, 3$

E [H_{τ_{i}}^{i}] \leq h_{b} (δ) + (δ + \frac{P_{e}}{δ}) {log}_{2} M_{i} .

Proof:

We prove the lemma for the case $i = 1$ . The proof for $i = 2, 3$ follows from a similar argument. For $i = 1$ , we obtain

	$E [H_{τ_{1}}^{1}]$	$= P {P_{e} (Y^{τ_{1}}, W_{2}) > δ} E [H_{τ_{1}}^{i} \| P_{e} (Y^{τ_{1}}, W_{2}) > δ] + P {P_{e} (Y^{τ_{1}}, W_{2}) \leq δ} E [H_{τ_{1}}^{1} \| P_{e} (Y^{τ_{1}}, W_{2}) \leq δ]$
		$\leq P {P_{e} (Y^{τ_{1}}, W_{2}) > δ} {log}_{2} M_{1} + P {P_{e} (Y^{τ_{1}}, W_{2}) \leq δ} E [H_{τ_{1}}^{i} \| P_{e} (Y^{τ_{1}}, W_{2}) \leq δ] .$		(17)

Note that the event ${P_{e} (Y^{τ_{1}}, W_{2}) > δ}$ implies that $τ_{1} = T$ , and $P_{e} (y^{τ_{1}}, W_{2}) > δ$ for all $0 \leq n \leq T$ . Hence, this event is included in the event ${P_{e} (Y^{T}, W_{2}) > δ}$ . Thus, applying Markov inequality gives

P {P_{e} (Y^{τ_{1}}, W_{2}) > δ} \leq P {P_{e} (Y^{T}, W_{2}) > δ} \leq \frac{P_{e}}{δ} .

As a result of the above argument, the right-hand side of (17) does not exceed the following

\frac{P_{e}}{δ} {log}_{2} M_{1} + E [H_{τ_{1}}^{1} | P_{e} (Y^{τ_{1}}, W_{2}) \leq δ] .

From Fano’s inequality we obtain

E [H_{τ_{1}}^{1} | P_{e} (Y^{τ_{1}}, W_{2}) \leq δ] \leq h_{b} (δ) + δ {log}_{2} M_{1} .

The proof is complete from the above inequality. ∎

As a result of the above lemma and (16), the inequality $E [τ_{i}] \geq \frac{log M_{i}}{I_{L}^{i} + ϵ} - \frac{h_{b} (δ)}{I_{L}^{i} + ϵ}$ holds. Finally, combining this inequality with (13)-(15) completes the proof of the theorem.

Iv-B An Alternative Proof for the Upper-Bound

In this part of the paper, we provide a series of Lemmas that are used to prove the Theorem. Define the following random processes.

Lemma 7.

For an $(M_{1}, M_{2}, N)$ -VLC with probability of error $P_{e}$ the following inequality holds

E [H_{T}^{i}] \leq h_{b} (P_{e}) + P_{e} {log}_{2} (M_{1} M_{2} - 1), for i = 1, 2, 3.

Proof:

The proof follows from Fano’s Lemma as in [2]. ∎

Lemma 8.

There exists $ϵ > 0$ such that, if $H_{t}^{i} \leq ϵ$ , then

	$E [log H_{t + 1}^{1} - log H_{t}^{1} \| F_{W_{2}} \times F_{t}]$	$\geq - (D_{1} + ϵ),$
	$E [log H_{t + 1}^{2} - log H_{t}^{2} \| F_{W_{1}} \times F_{t}]$	$\geq - (D_{2} + ϵ),$
	$E [log H_{t + 1}^{3} - log H_{t}^{3} \| F_{t}]$	$\geq - (D_{3} + ϵ)$

holds almost surely, where $D_{i}, i = 1, 2, 3$ are defined in (11).

Proof:

The proof is given in Appendix C. ∎

Lemma 9.

For $i = 1, 2, 3$ , define random process ${Z_{t}^{(i)}}_{t \geq 1}$ as

	$Z_{t}^{(i)} =$	$(\frac{log H_{t}^{i} - log ϵ}{D_{i}} + t + f_{i} (log \frac{}{H_{t}^{i}} ϵ)) 1 {H_{t}^{i} \leq ϵ}$
		$+ (\frac{H_{t}^{i} - ϵ}{I_{L}^{i}} + t) 1 {H_{t}^{i} \geq ϵ}$		(18)

where the function $f_{i}$ is defined as $f_{i} (y) = \frac{1 - e^{- μ_{i} y}}{D_{i} μ_{i}} .$ Then, there exists $μ_{i} > 0$ such that $Z_{t}^{(i)}$ is a submartingale w.r.t $F_{W_{1}} \times F_{W_{2}} \times F_{t}$ .

Proof:

Suppose $W_{2} = m$ for some $m \in [1 : M_{2}]$ . Given this event and using the same argument as in the proof of Theorem 1 in [2] we can show that $Z_{t}^{(i)} | W_{2} = m$ is a submartingale for all $m$ . More precisely, the inequality

E {Z_{t}^{(i)} - Z_{t + 1}^{(i)} | F_{W_{1}} \times F_{W_{2}}} \leq 0,

holds almost surely w.r.t $F_{W_{1}} \times F_{W_{2}}$ . Taking the expectation of the both sides in the above inequality gives

E {Z_{t}^{(i)} - Z_{t + 1}^{(i)}} \leq 0, \forall t \geq 0, i = 1, 2, 3.

Thus, $Z_{t}^{(i)}$ is a submartingale for $i = 1, 2, 3$ and w.r.t $F_{W_{1}} \times F_{W_{2}} \times F_{t}$ . ∎

Corollary 2.

Suppose $α_{1}, α_{2}, α_{3}$ are non-negative numbers such that $α_{1} + α_{2} + α_{3} = 1$ . Define $Z_{t} = α_{1} Z_{t}^{(1)} + α_{2} Z_{t}^{(2)} + α_{3} Z_{t}^{(3)}$ . Then, $Z_{t}$ is a submartingale w.r.t $F_{W_{1}} \times F_{W_{2}} \times F_{t}$ .

The Theorem follows from the above lemma, and the proof is given in Appendix D.

V The Shape of the Lower and Upper Bounds

In this Section, we point out a few remarks on $E_{u} (R_{1}, R_{2})$ and the lower-bound $E_{l} (R_{1}, R_{2})$ defined in Theorem 1. Furthermore, we provide an alternative representation for the bounds and show that the lower and upper-bounds match for a class of MACs.

We first compare the lower bound in (9) and the upper-bound in Corollary 1. For a given arbitrary rate pair $(R_{1}, R_{2})$ inside the feedback-capacity of a given MAC, consider a sequence of VLCs with rates $(R_{1}, R_{2})$ and with average probability of error approaching zero. Then, the following holds:

lim ϵ \to 0 lim P_{e} \to 0 lim M_{1} M_{2} \to \infty \frac{E_{u} (R_{1}, R_{2})}{E_{l} (R_{1}, R_{2})} = \frac{D_{u}}{D_{l}}

As a result of the above remark, it is concluded that for small enough probability of error, the bounds are different only in the constants $D_{u}$ and $D$

	$D_{1} (x_{1}, x_{2} \| \| z_{1}, z_{2})$	$= D_{Q} (x_{1}, x_{2} \| \| z_{1}, x_{2})$
	$D_{2} (x_{1}, x_{2} \| \| z_{1}, z_{2})$	$= D_{Q} (x_{1}, x_{2} \| \| x_{1}, z_{2})$
	$D_{3} (x_{1}, x_{2} \| \| z_{1}, z_{2})$	$= D_{Q} (x_{1}, x_{2} \| \| z_{1}, z_{2}) .$

On The Reliability Function of Discrete Memoryless Multiple-Access Channel with Feedback

Related Research

On the Reliability Function for a BSC with Noiseless Feedback at Zero Rate

Upper Bounds on the Feedback Error Exponent of Channels With States and Memory

Distributed Capacity of A Multiple Access Channel

Finite-Blocklength and Error-Exponent Analyses for LDPC Codes in Point-to-Point and Multiple Access Communication

Zero-rate reliability function for mismatched decoding

Reinforcement Learning Evaluation and Solution for the Feedback Capacity of the Ising Channel with Large Alphabet

The DNA Storage Channel: Capacity and Error Probability

I Introduction

Ii Problem Formulation and Definitions

Definition 1.

Definition 2.

Definition 3.

Ii-a The Feedback-Capacity Region of MAC

Definition 4.

Fact 1.

Ii-B Notational Conventions

Iii A Lower-Bound for the Reliability Function

Stage 1

Stage 2

Decoding

Theorem 1.

Proof:

Iv An Upper-bound for the Reliability Function

Theorem 2 (Upper-bound).

Corollary 1.

Proof:

Iv-a Proof of the Upper-Bound

Lemma 1.

Lemma 2.

Lemma 3.

Proof:

Lemma 4.

Proof:

Lemma 5.

Lemma 6.

Proof:

Iv-B An Alternative Proof for the Upper-Bound

Lemma 7.

Proof:

Lemma 8.

Proof:

Lemma 9.

Proof:

Corollary 2.

V The Shape of the Lower and Upper Bounds