An Upgrading Algorithm with Optimal Power Law

04/02/2020

∙

by   Or Ordentlich, et al.

∙

0

∙

share

Consider a channel W along with a given input distribution P_X. In certain settings, such as in the construction of polar codes, the output alphabet of W is `too large', and hence we replace W by a channel Q having a smaller output alphabet. We say that Q is upgraded with respect to W if W is obtained from Q by processing its output. In this case, the mutual information I(P_X,W) between the input and output of W is upper-bounded by the mutual information I(P_X,Q) between the input and output of Q. In this paper, we present an algorithm that produces an upgraded channel Q from W, as a function of P_X and the required output alphabet size of Q, denoted L. We show that the difference in mutual informations is `small'. Namely, it is O(L^-2/(|X|-1)), where |X| is the size of the input alphabet. This power law of L is optimal.