A Variational Bayes Approach to Decoding in a Phase-Uncertain Digital Receiver

07/04/2011 ∙ by Arijit Das, et al. ∙ Trinity College Dublin 0

This paper presents a Bayesian approach to symbol and phase inference in a phase-unsynchronized digital receiver. It primarily extends [Quinn 2011] to the multi-symbol case, using the variational Bayes (VB) approximation to deal with the combinatorial complexity of the phase inference in this case. The work provides a fully Bayesian extension of the EM-based framework underlying current turbo-synchronization methods, since it induces a von Mises prior on the time-invariant phase parmeter. As a result, we achieve tractable iterative algorithms with improved robustness in low SNR regimes, compared to the current EM-based approaches. As a corollary to our analysis we also discover the importance of prior regularization in elegantly tackling the significant problem of phase ambiguity.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Communication Systems face uncertainties of the transmitter and receiver oscillators, the noisy dispersive nature of the channel, among others for transferring messages from the transmitter to the receiver [11]. Incorrect alignment of the transmitter and receiver oscillators shows itself as a rotation of the received data on the complex plane and renders decoding impossible. The process of aligning these oscillators is known as synchronization and is an important aspect in the process of decoding [2].

Current state of the art in dealing with this problem is a joint iterative estimation of the phase and the symbol sequence, in an algorithm known as turbo-synchronization


. This algorithm is based on the framework of an EM-algorithm where probability distribution on the symbol decisions are used recursively to improve the estimate of phase, and vice-versa. This use of probability distributions over the possible symbols while making a decision, defined as a ”soft” decision, was the novelty in this approach

[3]. The algorithm could also be defined under other frameworks, such as one based on the sum-product algorithms [4], [7].

The EM-algorithm however, considers certainty-equivalence measures on the parameter values which lead to the loss information about their statistical uncertainty. This work aims to generalize this framework into a fully Bayesian treatment, which is flexible enough to ”learn” the varied channel conditions along with the uncertainties associated with the modelled parameters. This was however partially achieved in [9], wherein the authors used a uniform prior over the phase. Good Bayesian learning models take into consideration all the statistical information, the data has to offer and therefore advice better decisions under adverse conditions.

This paper presents the advantage of using a generalization of the uniform distribution on the unit circle, the Von-Mises distribution in resolving phase ambiguity. Literature consists of either using a huge number of pilot symbols or modified likelihood estimators

[4] which are generally inefficient with respect to power and time, to deal with the problem. In the course of our analysis, we find a natural elegant solution to the problem, which does not add any computational burden to the synchronization routine and exalts the importance of prior regularization in Bayesian Analysis.

As constellation alphabets belong to finite fields, the receiver data models are inevitably mixture distributions which are not in the exponential family. Finite dimensional sufficient statistics [5]

and conjugate prior distributions

[1] are a unique characteristic to the exponential family. They are important in devising recursive learning algorithms. Unfortunately as this is not the case, it leads to models with substantially expensive inference procedures. A deterministic approximation method, known as Variational Bayesian (VB) Method [14], is applied in this paper to deal with the problem.

The rest of the paper is structured as follows. Section II defines the system model encompassing unknown phase and symbol sequence. Section III describes the single symbol transmission inference. Section IV describes the combinatorial complexity, the Variational Bayes Method and how it is applied to the Multi-Symbol transmission inference. In section V, numerical results for the performance of the described receiver and its comparisons with the current state of the art are provided. Section VI discusses the results and its implications. Section VII presents the concluding remarks and future course of actions.

Ii System Model

Structured redundancy is usually added in the transmitted data [2], in a process known as coding, to protect from the noisy nature AWGN channel. Equal partitions of the binary message stream formed from uniformly sampled and quantized analog signals, are bijectively mapped to a complex symbol from a constellation alphabet of size M. These symbols are then modulated onto a carrier signal which can be sent through a physical channel. On the receiver side, the reverse operation called demodulation attempts to decode or identify the transmitted symbol sequence, which are now corrupted by noise [3].

We observe the data vector

during every symbol period with with

and assume the prior probabilities corresponding to the symbol vector

to be . represents the phase difference of the received signal to the transmitted, with noise in an AWGN channel [10]. Ease of inferential analysis dictates the augmentation of the model using a latent variable. Let it be , where and is the Kronecker Delta. Hence is a vector pointer to the component active at the symbol period. The data generation model for such a symbol transfer in a single symbol period can be defined as:


where : Complex Normal and : Multinomial distributions, , , with as the pulse sequence and

the carrier digital frequency which are assumed to be known. In this analysis we assume known SNR, based on which we can calculate the noise variance,


a) Prior Distribution to Phase: Von-Mises Distribution

As we will see later, the VB approximation on the joint distributions lead to marginal distributions of the phase from the exponential family, hence the use of conjugate priors remains important. Von-Mises distribution

[12], [8] is a useful distribution over the unit circle defined from on the real line, which serves as a conjugate prior to the phase for our conditional data model. It has one complex parameter, which serves as both location and concentration parameter for the distribution and has been used in varied applications on directional data.


where . In the signal processing literature this distribution is also known as the Tikhonov distribution [12] and has been previously used to model the phase error from the estimated phase using a phase-locked loop (PLL). In this work Von-Mises/Tikhonov distribution is considered to be the prior distribution of the phase in a Bayesian framework which was explicitly introduced in [10]. As the Von-Mises/Tikhonov distribution is confined to the unit circle, means and variances outside

are undefined. Hence circular moments are used which are defined as:


Limiting behaviour of this distribution under various conditions of its hyperparameter are as follows



is the normal distribution and

is the uniform distribution.

Iii Single Symbol Transmission

Now we present the inference procedure for the single symbol transmission, where exact inference was shown to be possible in [10]. This analysis forms the basis for our generalization into the multi-symbol cases, wherein the Variational Bayes Method is applied. The data generation model in this case is taken to be



Fig. 1: Graphical Model - Single-symbol model in the case of uncertain phase

The marginal posterior distributions for phase and the symbol can be easily calculated in this case and are given in [10] to be




The mean value of for each component is given by . Also

can be characterized as a Distrete Time Fourier Transform on the incoming data, which is intuitive since

is a sufficient statistic to the phase . Here however we get more information than from a simple DTFT, namely about the confidence over the expected phase.

Another important observation about can be seen with the following algebraic expansion:


From the above equation we can see that when (Uniform Prior), all except the first term vanishes. That implies then that when the prior is a uniform distribution, the decoding distribution is invariant to the symbols on the same radius in a QAM constellation, which is the origin of phase ambiguity. However a non-zero makes depend on the value of the symbols and hence results in a unique MAP from the decoding distribution.

Iv Multi-Symbol Transmission

Real world communication systems involve multiple symbol periods, which are observed sequentially and a decoding distribution for the symbols is the desired output. The joint data model under the independent symbol sequence assumption (reasonable in turbo processing where pseudorandom inter-leavers are typically employed [9]) is then


As a consequence of this product over a summation, term mixture model appears after time periods, a combinatorial explosion which leads to intractable exact inference.

a) Variational Bayes Method

A deterministic approximation, known as the Variational Bayes Method is employed to deal with this computational intractability. The Method aims at minimizing the Kullback-Leibler Divergence between the candidate approximation and the original joint distribution. This divergence can be interpreted as an information difference

[15] between the two distributions. The intuition then is to get a functional form of the posterior marginals, so as to minimize the information lost due to the forced independence [13]

. These methods have found many applications in machine learning, artificial intelligence and more recently in signal processing

[9]. The essential result which forms the basis of this method is as follows:

Theorem IV.1

Variational Bayes Method [14]: Let be the posterior distribution of multivariate parameter, . The latter is partitioned into q sub-vectors of parameters or q-nodes:


Let be an approximate distribution restricted to the set of conditionally independent distributions for ,,…:


Then, the minimum of Kullback-Leibler divergence is reached for


where and in , and . Now is the VB-aproximation, and are the VB-marginals.

b) Offline VB Approximation

Consider the situation, where we observe a batch of symbol transfer periods together and are required to infer the marginal distributions for all the transferred symbols and the phase parameter. Prior independence of the symbols is assumed. However it should be noted, that aposteriori the symbols are independent conditional on . Expanding the joint probability, over all unknown parameters, we have



Fig. 2: Graphical Model - Multi-symbol model in the case of uncertain phase

From the above joint distribution, we find the posterior distributions for the label field and the phase. Integrating out the label field entails a summation, making it intractable [13]. We therefore resort to using Theorem II.1 and derive the approximate posterior distributions of the parameters as


with the shaping parameters as


and the associated VB moment as


again can be characterized as a DTFT. Important subsets of this method discussed in the literature include [9] where the authors have assumed an uniform prior to the phase which implies . Further if we make the ratio

we get the EM-algorithm based turbo-synchronization routine as discussed in [3].

c) Online VB Approximation

In this case we observe each symbol transfer period sequentially and are required to infer the marginal distributions for each transferred symbol and phase parameter, after each symbol period. Again prior independence of the symbols is assumed. To deal with the increasing combinatorial complexity after each time step we conflate the posterior distribution of which is a mixture of Von-Mises components using Variational Bayes Method in to a single Von-Mises posterior. The associated sufficient statistic can then be used as the prior for the next symbol period. Expanding the joint probability, over unknown parameters after each time step, we have




On equation (13) from Section III of the posterior distribution of the phase, we apply the VB Method and derive the approximate conflated posterior distribution as


with the shaping parameter as


Using this shaping parameter as initialization for the following symbol period we get from the single symbol transmission case:




again can be characterized as a DTFT as earlier.

V Simulation

Fig. 3: - Performance of the VB-based decoding algorithms in the multi-symbol case

The above figure describes the proportion of successful identifications of the symbol transferred for various values of SNR, using the methods developed in this paper. In each batch 20 symbols are transmitted through an AWGN channel. In the low-SNR regime, from -15dB to 5dB we see that the Bayesian algorithms are significantly better. Independent decoding algorithm uses the exact decoding algorithm over each subsequent symbol period ignoring the previous periods. This method turns out to be the fastest and most accurate for this simulation. The Online VB and the Offline VB approximations provide almost as good results, though they would be more robust in a more general setting. We can also clearly see the performance of [9] and [3] is significantly lower with respect to the decoding success. It should be noted, to deal with the problem of phase ambiguity for these two methods, 5 pilot symbols are used in front of the original 20.

Vi Discussion

We get two main results from the analysis performed in this paper. Primarily we are able to extend the EM-Algorithm based synchronization into a fully Bayesian synchronization procedure. This can form the basis for future generalizations and analysis of even more complicated scenarios. Secondly, as a corollary to our analysis we are able to resolve the problem of phase ambiguity.

As shown in the Section III, setting renders independent of the expected phase , which thereby results in the apparent ambiguity in the decoding distribution. The notion of non-zero seems quite inconsequential, but a concentrated prior however informs the presence of a unimodal phase distribution and regularizes the observation model which is intrinsically rotationally invariant for QAM. It must be noted however, that as the number of incoming observations increases the sampling distribution of the MAP symbol becomes relatively invariant to the value of , as long as .

Computationally it presents no overhead, as it involves only the addition of a constant. In contrast past approaches like Differential Coding [2], angle differential QAM scheme [6] and the use of pilot symbols [3] [4], among others which have been proposed to deal with this rotational invariance are visibly more complex and lack the simplicity of this observation.

In [3] the authors use the EM-Algorithm to deal with phase synchronization. This essentially implies using the sifting property of the Kronecker Delta function and thus using the expectations of the parameters in the associated functions rather than the expectations of the functions of the parameters itself. Significantly lower accuracy of the decoding distribution in the low-SNR regime is observed which converges in performance with increasing SNR.

Vii Conclusions

In this paper we have been able to extend the EM framework for tackling the phase-synchronization problem, to a fully Bayesian VB based algorithm wherein as a virtue of its modelling ability the problem of phase ambiguity does not arise. This makes the concept of using ”soft” information truly complete as no statistical information is lost at any point in the iterations. Further the basic framework developed motivates further relaxations into more general channels, such as those with multi-path fading, like Rician Fading [2]. In future work we would also be relaxing the condition on known noise parameter and deal with it in a Bayesian framework.

Viii Acknowledgments

This research was supported by Science Foundation Ireland, grant 08/RFP/MTH1710


  • [1] Persi Diaconis and Donald Ylvisaker. Conjugate Priors for Exponential Families. The Annals of Statistics, 7(2):pp. 269–281, 1979.
  • [2] Andrea Goldsmith. Wireless Communications. Cambridge University Press, New York, NY, USA, 2005.
  • [3] C. Herzet, N. Noels, V. Lottici, H. Wymeersch, M. Luise, M. Moeneclaey, and L. Vandendorpe. Code-Aided Turbo Synchronization. Proceedings of the IEEE, 95(6):1255 –1271, june 2007.
  • [4] C. Herzet, K. Woradit, H. Wymeersch, and L. Vandendorpe. Code-Aided Maximum-Likelihood Ambiguity Resolution Through Free-Energy Minimization. Signal Processing, IEEE Transactions on, 58(12):6238 –6250, Dec 2010.
  • [5] Christian Hipp. Sufficient Statistics and Exponential Families. The Annals of Statistics, 2(6):pp. 1283–1292, 1974.
  • [6] Jeng-Kuang Hwang, Yu-Lun Chiu, and Chia-Shu Liao. Angle differential-QAM scheme for resolving phase ambiguity in continuous transmission system. Int. J. Commun. Syst., 21:631–641, June 2008.
  • [7] Michael I. Jordan. An introduction to variational methods for graphical models. In Machine Learning, pages 183–233. MIT Press, 1999.
  • [8] C. G. Khatri and K. V. Mardia. The Von Mises-Fisher Matrix Distribution in Orientation Statistics. Journal of the Royal Statistical Society. Series B (Methodological), 39(1):pp. 95–106, 1977.
  • [9] Mauri Nissila and Subbarayan Pasupathy. Adaptive iterative detectors for phase-uncertain channels via variational bounding. Trans. Comm., 57:716–725, March 2009.
  • [10] Anthony Quinn, Jean-Pierre Barbot, and Pascal Larzabal.

    The Bayesian Inference of Phase

    In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 2011, ICASSP’11, 2011.
  • [11] C.E. Shannon. Communication in the presence of noise. Proceedings of the IEEE, 72(9):1192 – 1201, sept 1984.
  • [12] Yuriy S. Shmaliy. Von Mises/Tikhonov-based distributions for systems with differential phase measurement. Signal Processing, 85(4):693 – 703, 2005.
  • [13] V. Smidl and A. Quinn. The Variational Bayes Approximation In Bayesian Filtering. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2006,  3, page III, May 2006.
  • [14] Vaclav Smidl and Anthony Quinn. The Variational Bayes Method in Signal Processing. Springer, Dublin, Ireland, 2006.
  • [15] Martin J. Wainwright and Michael I. Jordan. Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends in Machine Learning, 1:1–305, 2008.