I Introduction
There is an ever increasing demand for higher data rates from communication systems. This has led to a renewed interest in the communication research community to analyze systems with feedback in order to achieve more reliable communication. Shannon in [10]
showed that use of feedback in communication systems does not improve its capacity, but feedback can simplify encoding and decoding at the receiver higher reliability, along with improving the error probability performance through higher error exponents. With the feedback of the received symbols, the transmitter has a copy of the version of the message that was estimated at the receiver. This helps the transmitter in encoding the original message and repeatedly sending it to the receiver for more reliable estimation.
Most of the previous work have provided asymptotic results for the noisy feedback paper. The definition of capacity as the code rate that achieves zero error in a noisy channel with infinite blocklength was given by Shannon in [10, 9]. In 1966, Schalkwijk and Kailath proposed a linear scheme (sk scheme) which utilizes the feedback to frame the subsequent symbols and showed that it achieves a superior error performance [8] compared to conventional systems without feedback. For noisy feedback channels, the authors in [5, 6] provided the asymptotic results for the error performance. Kim et al in [4]
showed that nonlinear codes parametrized by recurrent neural network outperform existing linear schemes for AWGN channels with noisy output feedback. Jiang et al in
[3]provided a recurrent neural network based turbo autoencoder to use the feedback in order to frame the subsequent transmitted codewords. Chance and Love in
[1] proposed a linear scheme for noisy feedback channels and showed that their scheme achieves improved error performance than the sk scheme for noisy channels. While conjectures on optimal linear codes exist [1], the optimal linear code is unknown for channels with noisy output feedback. In this paper, we aim to bridge this gap. We apply the sequential decomposition framework to analyze a general discrete memoryless channel with noisy feedback, presented in [11], to awgn channels with noisy feedback. We propose a novel methodology based on dynamic programming to obtain optimal policies for Gaussian channels with noisy feedback. Our main contributions are as follows:
We propose a family of linear sequential codes that are equipped with an optimal decoder, i.e., Kalman filter, for awgn channels with noisy feedback (Section
III). 
We derive the close form solution for the optimal linear sequential code via dynamic programming. To do so, we introduce a novel mdp framework that uses variances in the estimation at the transmitter and receiver as states (Section IV).

We show that the sk scheme is optimal (in minimizing probability of error) among all linear sequential codes for noiseless feedback channels (Corollary 1 in Section IV).

We characterize the minimum error that the optimal codes achieve as a function of the number of transmissions, in a closed form solution (Theorem 1 in Section IV). We observe that the variance in the estimate of the message approximately drops exponentially for noiseless feedback and polynomially for noisy feedback settings (Section V).
Ii Gaussian Channel with Feedback
We consider a discrete time pointtopoint Gaussian memoryless channel with output feedback. The transmitter and the receiver are connected through an awgn forward channel and an awgn feedback channel as shown in Fig. 1. The transmitter intends to send a message to the receiver where
is Gaussian distributed as
. The forward and the feedback transmissions can be expressed as(1) 
where is the transmitted signal, is the received signal after the forward transmission, and is the received feedback at the transmitter. and are the realizations of the forward and the feedback Gaussian noises and respectively.
Let be the series of transmissions from the transmitter to the receiver successively over the forward pass of channel excluding the original transmission and be the corresponding feedback symbols received at the transmitter. In this paper, we intend to find the length sequence of transmissions of ’s as a function of the feedback ’s and the message that would improve the error performance of the estimation at the receiver. The channel inputs are constrained to some given power constraint as at each instant . Precisely, we aim to find a pair of encoder mapping and decoder mapping that minimizes , where the randomness is from the additive noise and the message .
We introduce a state variable at the encoder other than the transmitted symbol to keep track of the symbols generated at each instant from the encoder. At each instant, we devise a strategy which generate the next transmitted symbol from the original message, the previous state, the received feedback. We can represent the whole operation as
(2)  
(3) 
At the receiver, we employ a Kalman Filter for the linear estimation of the symbols that are received. Our goal is to reduce the error in the the estimation of the symbol at the receiver i.e. to reduce given as
(4) 
Iia The SchalkwijkKailath Scheme
In this section, we present the sk scheme as a variant of the Elias scheme described in [2] and in the next section we show that sk scheme can be derived as a special case using our proposed dynamic programming algorithm.
The first transmission from the encoder is assumed to be the message itself as distributed as . The transmitted signal is a scaled version to meet the power constraint where and signal that is transmitted eventually is given as . The received signal and the corresponding noiseless feedback at the transmitter are given as
(5) 
where is the additive white Gaussian noise. The mmse estimate of the transmitted symbol at the receiver is given as . According to the sk scheme the eventual transmissions are chosen as
(6) 
where . Basically, at each instant the transmitter evaluates the error in the mmse estimate of the original symbol and sends the error in the next transmission such that
(7) 
Thus, from (7), we can deduce the estimate of the original symbol , i.e. to be with the error variance for each transmission is given as
(8) 
Iteratively, it can be shown that after transmissions, the original variance in the estimate of is reduced to
(9) 
Iii Sequential linear codes for awgn channels with feedback
In this section, we introduce a family of sequential linear codes that specialize the framework developed in [11] to a point to point awgn channel with noisy feedback. The encoder combines the message, feedback and previous symbols linearly in order to generate the subsequent symbols. The decoder employs Kalman filter to estimate the message, which achieves the mimimum mean square error for a given linear encoder under AWGN channels with feedback. In this section, we introduce parameters and formulation of our proposed encoder and decoder. In Section IV, we show that the optimal linear sequential code can be learned via dynamic programming.
Iiia Sequential Linear Encoding at Transmitter
We consider a linear sequential encoder with hidden state as depicted in Figure 1. The state is updated linearly as a function of the original symbol , the past state and the immediate feedback , i.e.,
(10) 
The transmitted signal is a scaled version of , i.e.,
(11) 
where is computed in order to ensure the maximum (peak) power constraint, i.e., where .
We show in Section IV that the optimal values for that minimize the estimation error can be learned via dynamic programming.
IiiB Estimation via Kalman Filter at Receiver
The estimation at the receiver is done through a Kalman Filter. We first define as where is the true message, is a state variable that tracks the past symbols at the transmitter, and is the signal received at the receiver. From equations in (1), (10) and (11), we can easily show that satisfies the following linear equations
(16a)  
(16b) 
with , and , where
(17)  
(18) 
and the matrices in (16a) and (16b) are defined as
(19) 
and
(20) 
With the assumption that the initial value of the state variable , the initial state . Kalman filter allows one to estimate the linear mmse estimate based on in a recursive manner [7]. We let , , and denote the linear mmse estimate of , , and given , respectively. (Since and ’s are independent Gaussian, linear mmse estimate of is the mmse estimate of .) Let denote the covariance of the estimation error given the received signal :
(21)  
(22) 
where the individual terms in (22) are defined as follows.
(23)  
(24)  
(25) 
Now from the standard solution to Kalman filters from [7], we obtain recursive solutions to the error covariance as
(26)  
(27)  
(28) 
We described the encoding at the transmitter and the estimation at the receiver for a single step at any instant . This process is repeated for transmissions from to till we reach at the desired at time .
For any given (linear sequential) encoder, the optimal decoder can be easily obtained by Kalman filter. Can we find the optimal encoder within the family of linear sequential codes for noisy feedback settings? Our family of codes includes SK scheme as a special case. Is SK scheme the optimal solution (within this family) for channels with noiseless feedback? The answers to these questions are affirmative. In the following, we derive the closedform optimal solution by dynamic programming and show that SK scheme is optimal among this family of sequential linear codes for noiseless feedback channels.
Iv Optimal linear sequential codes via dynamic programming
Let denote the linear sequential encoding operation at time , i.e., and , introduced in the previous section. In this section, we present the dynamic programming algorithm to derive the optimal coding scheme that minimizes the error variance at the receiver.
We treat the step process as a mdp with the encoder function to be the control action at every instant . We construct a MDP with Markovian state at time defined as , where and so that we track the variances (14)  (15) at the sender and (21)  (25) at the receiver, respectively.
The intended messages and the received feedback are encoded and sent repeatedly to the receiver over certain number of iterations through the time index . is the time index for the last transmission while is the zeroth transmission where there is no encoding of the transmitted symbol. While denote the computed optimal policies, is the error variance (return) that will be obtained if we follow the optimal policy from the current state for the remaining transmissions. The immediate cost function is defined to be the value function at the next state represented as
(29) 
where is the transition function. We can summarize the transition function in Algorithm 1.
The bellman update for the value function is given as
(30)  
(31) 
where is the optimal linear controller action at time , and is the optimal linear update of . Given that a total of transmissions are going to be sent, the value function of any state at any time represents the message variance at the receiver if we start from that state for transmitter and receiver and take the optimal actions for transmissions. denotes the return for the last step if the last optimal policy was played to generate the last iteration . The value of the value function for different states gives the value of the estimated variance when we don’t have any more transmissions and we know the error variance matrix. It is straightforward value from the matrix . The proposed dynamic program can be summarized as follows.

, , .

For , and ,
(32)
We solve the dynamic program above and obtain the closed form expressions for the value function and the optimal strategy for every as shown in the following.
Theorem 1.
The value function and the optimal strategy at any time are as follows:
(33)  
(34) 
where is the number of encoded transmissions that are remaining and is given as , and
(35)  
(36) 
where , for
Before we prove Theorem 1, we show the following.
Corollary 1.
Proof.
We show that the optimal policy derived in Theorem 1 coincides with the SK scheme for noiseless case. For a noiseless feedback channel we have and i.e.
(37)  
(38) 
where the last equality holds since and . Note that for noiseless feedback scenarios. ∎
Proof of Theorem 1.
Let us derive the expression for in the first iteration i.e. . Assuming we want to compute the values for some given and , we use the function in Algorithm 1 to find and . We can find the closed form expression for using any symbolic toolbox to be some function .
It is worth noting that we assume as original message is involved in encoding only in the original transmission. Also, we can assume as it can be adjusted through scaling. We then minimize the expression to find through which leads to
(39) 
By substituting as in (32), we get
where
(40) 
This process is repeated for and we get similar expressions as above with replaced with where
∎
V Estimation Error
Let us now visualize the estimation error that we get by using our proposed linear scheme as a function of the total number of transmissions . Our proposed dynamic programming helps us to determine the value of the estimation error in closed form. In fact, the value function gives the value of that we end up with after the remaining transmissionss if we follow the optimal encoding policy till the transmission.
After the initial transmission of the original message , we can find the values of and from below.
We can substitute these values in to get , where
(41) 
The value of can be obtained from (35), (36) and (40). We plot the value in the Fig. 3 as the mse in dB in the estimation of the message . We consider , , and varying values of . We see that the estimation error variance drops linear in logarithmic terms where as for all the other cases it is polynomial.
Vi Conclusions
We apply a novel dynamic programming approach to design linear codes for AWGN channels with noisy output feedback. We start with a family of linear sequential codes and characterize the optimal code within this family (and its estimation error) by dynamic programming in a closed form. We visualize the estimation error as a function of number of transmissions for channels with various levels of noisy feedback. For noiseless settings, the optimal code we derive coincides with the celebrated SK scheme. One big open question is whether the optimal codes we devised via dynamic programming are optimal within the whole family of linear codes. We conjecture the answer is affirmative. Proving (or disproving) the conjecture is left as future work. On a related note, Chance and Love [1] has a conjecture on optimal linear schemes for noisy feedback settings. They start with the most general linear family of codes, but their conjectured optima solution boils down to , which belongs to our family of linear sequential codes. Comparison between our work and their work is not straightforward as our power constraint is different from theirs (peak vs. average power constraint). Extending our work to the average power constraint setting is another interesting open problem.
References
 [1] (2011) Concatenated coding for the AWGN channel with noisy feedback. IEEE Transactions on Information Theory 57 (10), pp. 6633–6649. External Links: Document, 0909.0105, ISSN 00189448 Cited by: §I, §VI.
 [2] (201001) Variations on a theme by Schalkwijk and Kailath. IEEE Transactions on Information Theory 56 (1), pp. 6–17. External Links: Document, 0812.2709, ISSN 00189448 Cited by: §IIA.

[3]
(201911)
Turbo Autoencoder: Deep learning based channel codes for pointtopoint communication channels
. External Links: 1911.03038, ISSN 23318422, Link Cited by: §I.  [4] (2018) Deepcode: feedback codes via deep learning. In Advances in Neural Information Processing Systems, Vol. 31, pp. 9436–9446. Cited by: §I.
 [5] (2011) Error exponents for the gaussian channel with active noisy feedback. IEEE Transactions on Information Theory 57 (3), pp. 1223–1236. External Links: Document, ISSN 00189448 Cited by: §I.
 [6] (2007) The Gaussian Channel with Noisy Feedback. IEEE International Symposium on Information Theory  Proceedings, pp. 239–265. External Links: Document, ISBN 1424414296 Cited by: §I.
 [7] (1986) Stochastic Systems: Estimation, Identification, and Adaptive Control. Technical report External Links: ISBN 013846684X Cited by: §IIIB.
 [8] (1966) A Coding Scheme for Additive Noise Channels with Feedback—Part I: No Bandwidth Constraint. IEEE Transactions on Information Theory IT12 (2), pp. 172–182. External Links: Document, ISSN 15579654 Cited by: §I.
 [9] (1948) A Mathematical Theory of Communication. Bell System Technical Journal 27 (3), pp. 379–423. External Links: Document, ISSN 15387305 Cited by: §I.
 [10] (1956) The zero error capacity of a noisy channel. IRE Transactions on Information Theory 2 (3), pp. 8–19. External Links: Document, ISSN 21682712 Cited by: §I, §I.
 [11] (202002) Sequential decomposition of discrete memoryless channel with noisy feedback. External Links: 2002.09553, ISSN 23318422, Link Cited by: §I, §III.
Comments
There are no comments yet.