I Matrix Construction
In Wireless communication system or Free Space Optical communication system, due to the shadow effect or the turbulent of environment, the channel conditions can be flipped from “good” to “bad” or “bad” to “good” state such as Markov model after the transmission time[mcdougall2003sensitivity] [wang1995finite]. For simple intuition, in “bad” channel, a signal will be transmitted incorrectly and in “good” channel, the signal is received perfectly. Suppose a system has total channels, the “good” channel is noted as “1” and “bad” channel is “0”, respectively, the transmission time between transmitter and receiver is
, the probability the channel is flipped after the transmission timeis . We note that if the system using a binary code such as On-Off Keying in Free Space Optical communication, then the flipped probability is equivalent to the error rate.
Consider a simple case for , suppose that at the beginning, both channel is “good” channel, the probability of system has both of channels are “good” after transmission time , for example, is . Let call is the probability of system from the state has “good” channels and “bad” channels transfers to state has “good” and “bad” channels. Obviously that and . For example, the transition matrix and for and are constructed respectively as follows:
These transition matrices are obviously size since the number of “good” channels can achieve discrete values from . Moreover, these class matrices have several interesting properties: (1) all entries in matrix can be determined by Proposition 1; (2) the inverse of matrix is given by Proposition 2. Moreover, this matrices are obviously central symmetric matrix.
For channels system, the transition matrix has size and all entries in row column will be established by
From the definition, is the probability from state has “good” channels or bit “1” transfer to state has “good” channels or bit “1”. Therefore, suppose is the number channels in “good” channels that is flipped to “bad” channels after the transmission time and . Thus, to maintain “good” channels after the time , the number of “bad” channels in “bad” channels must be flipped to “good” channels is:
Therefore, the total number of channels are flipped their state after transmission time is:
and the total number of channels that preserves their state after transmission time is . However, . Similarly, the number of “bad” channels in “bad” channels must be flipped to “good” channels should be in . Hence:
Therefore, can be determined by below form:
All the entries of inverse matrix given in Proposition 1 can be determined via original transition matrix for .
Due to the pages limitation, we will show the detailed proof at the end of this paper. To illustrate our result, an example of the inverse matrix are shown as follows:
Next, base on the existence of inverse matrix closed form, we will show that a capacity closed form for a discrete memory-less channel can be established. We note that in [cover2012elements], the authors said that haven’t closed form for channel capacity problem. However, with our approach, the closed form can be established for a wide range of channel with error rate is small.
Ii Optimize system capacity
A discrete memoryless channel is characterized by a channel matrix with and representing the numbers of distinct input (transmitted) symbols , , and output (received) symbols , , respectively. The matrix entry represents the conditional probability that given a symbol is transmitted, the symbol is received. Let
be the input probability mass vector, wheredenotes the probability of transmitting symbol , then the probability mass vector of output symbols , where denotes the probability of receiving symbol . For simplicity, we only consider the case such that the number of transmitted input patterns is equal the number of received input patterns. The mutual information between input and output symbolsis:
Thus, the mutual information function can be written as:
where denotes the th component of the vector . The capacity of a discrete memoryless channel associated with a channel matrix puts a theoretical maximum rate that information can be transmitted over the channel [cover2012elements]. It is defined as:
Therefore, finding the channel capacity is to find an optimal input probability mass vector such that the mutual information between the input and output symbols is maximized. For a given channel matrix , is a concave function in [cover2012elements]. Therefore, maximizing is equivalent to minimizing , and the capacity problem can be cast as the following convex problem:
Optimal numerical values of can be found efficiently using various algorithms such as gradient methods [grant2008cvx] [boyd2004convex]. However, in this paper, we try to figure out the closed form for optimal distribution via KKT condition. The KKT conditions state that for the following canonical optimization problem:
construct the Lagrangian function:
then for , , the optimal point must satisfy:
Our transition matrix that is already established in previous part can represent as a channel matrix. In the optical transmission, for example, the transmission bits are denoted by the different levels of energy, for example, in On-Off Keying code bit “1” and “0” is represented by high and low power level. This energy is received by a photo diode and converse directly to the voltage for example. However, these photo diode work base on the aggregate property when collecting all the incident energy, that said, if two channels transmit a bit “1” then the photo diode will receive the same energy “2” even though this energy comes from a different pair of channels. Therefore, the received signal is completely dependent to the number of bits “1” in transmission side. Hence, in receiver side, the photo diode will recognize states . From this property, the transition matrix is the previous section is exactly the system channel matrix. The channel capacity of system, therefore, is determined as an optimization problem in (1).
Next, we will show that the above optimization problem can be solved efficiently by KKT condition. We note that our method can establish the closed form for general channel matrix and then the results are applied to special matrix . First, we try to optimize directly with input distribution , however, the KKT condition for input distribution is too complicated to construct the first derivation. On the other hand, base on the existence of inverse channel matrix, the output variable is more suitable to work with KKT condition since. Due to , the Lagrange function from (2) with output variable is:
Using KKT conditions, at optimal point , , :
Because and , so always exist . From with , we can see clearly that with or with .
Therefore with fifth condition, with . Then, we have simplified KKT conditions:
The derivations are determined by:
Next, using derivation of I(X,Y) at and last condition:
Next, using first simplified condition, we have the sum of all output states is 1.
Therefore, can be figured out by:
From the second simplified condition, we can compute :
Due to the channel matrix is a closed form of , the optimal input vector and output vector also is a function of . However, we note that since the KKT condition works directly to the output variable , the optimal input can be invalid or . In next step, our simulations shown that for and , both output and input vector are valid. That said, our approach will be worked with a good system where the error probability is small. In case of the invalid optimal input vector, the upper bound of channel capacity, of course, will be established.
In this paper, our contributions are twofold: (1) establish an inverse closed form for a class of channel matrix based on the error probability ; (2) figure out the closed form for channel matrix with small error rate and determine the upper bound system capacity for a high error rate channel.