Analysis on the Empirical Spectral Distribution of Large Sample Covariance Matrix and Applications for Large Antenna Array Processing

03/08/2019 ∙ by Guanping Lu, et al. ∙ 0

This paper addresses the asymptotic behavior of a particular type of information-plus-noise-type matrices, where the column and row number of the matrices are large and of the same order, while signals are diverged and time delays of the channel are fixed. We prove that the empirical spectral distribution (ESD) of the large dimension sample covariance matrix and a well-studied spiked central Wishart matrix converge to the same distribution. As an application, an asymptotic power function is presented for the general likelihood ratio statistics for testing the presence of signal in large array signal processing.



There are no comments yet.


page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

The signal detection of information-plus-noise-type matrices is fundamental to modern wireless communication networks. Due to the growing scale of network and limited time resource, the available sample sizes cannot be quite larger than the dimensions. For this issue, the traditional covariance matrix theory is actually not applicable, which needs much larger sample sizes than signal dimensions[1]

. Thus, random matrix theory (RMT) has been used in resolving the high dimension estimation problems in signal processing


The work [9][11] studied the user detection and signal detection algorithms using large dimension signal-plus-noise matrices. It was shown that the random matrix theory on large dimension signal-plus-noise matrices could guide the large antenna array signal processing very well. On the other hand, the work [5] researched the empirical spectral distribution (ESD) convergence behavior of three large dimension sample covariance matrices (SCM) categories. Given a Hermitian matrix , for any real , the ESD, , is defined by


where denotes the cardinality of the set , is the

-th eigenvalue.

[6] and [3] investigated the general likelihood ratio test (GLRT) for linear spectral statistics of the eigenvalues of high-dimensional SCM from Gaussian populations. There is a gap between the traditional signal detection model as [11] and state-of-art random matrices theory [3], which, hopefully, can be connected by our research as a bridge.

In this work, a simple kind of information-plus-noise-type matrices with isotropic noise and limited signal dimension is studied. It is proved that the ESD of the received SCM converges in distribution to the ESD of central spike Wishart matrix[2]. Based on this feature, GLRT in [6] has been used in hypothesis test of the signal detection for massive MIMO systems. In this paper, the GLRT tests were applied in time domain to support the hybrid transmission scheme such as filter bank multicarrier (FBMC) or filtered multitone (FMT) modulation [7]. Except the scenario in this paper, the results could be used widely from MIMO detection[8] to multiuser detection[11].

By proposed hypothesis test method, we evaluate the detection performance of large antenna array using the same number of antennas but the different number of samples. it is found that the simulation results highly agree with those from the theoretical analysis.

In this paper, is the antenna number and is the sample number which are and in [5] and Lemma 1, is the n-th element of sequence ,

is the row vector,

means constructing a matrix by as rows, returns a square diagonal matrix with the elements of vector v on the main diagonal, is the Hermitian transpose operator, is the received signal matrix, is the -th element of which is in [5] and Lemma 1,

is the Normal distribution,

is the complex Normal distribution, and are complex number field and real number field, respectively.

Ii System Model

In signal detection, SCM analysis is used to explore the fundamental limits of communication[11]. Consider that there is a single transmitter which sends the signal to receivers (or antenna elements) in an -length channel as in [8]. For single antenna element of , the received signal is the summation of the tap delayed transmitted signal vector with propagation as , where . Accordingly, -length points are sampled from each antenna, which is written as


where is the concatenate factor of transmitter amplifier and channel, ,

is the transmit signal and follows the Binomial distribution as

or Normal distribution as .

is the noise vector at the receiver containing independent identically distributed (i.i.d.) complex entries and unit variance,

is the signal power on each taps. Suppose the receiver truncates length- signal for detection. The length of channel delay is not larger than samples. The of the signal vector has elements as .

The task is to detect whether the signal is presented by processing the SCM of the receiver. It turns to be a hypothesis testing problem, where means that the signal does not exist, and means that the signal exists. The received signal samples under the hypothesis test are given, respectively, as


Furthermore, it is assumed that the noise and channel propagation are uncorrelated. The complex Gaussian noise has the property as


The channel propagation has the same property.

In large scale antenna systems, the signal detection could be based on the joint operation of the sampled signal from each antenna element, such as in [8]. The sample vector from each antenna elements is formed by samples, and it could construct a matrix as


where has a dimension of with element as and ,,. The maximum channel length is a fixed value regardless the value of . The SCM of (5) is


Note as the ESD of the SCM. Under hypothesis , the SCM is not a good approximate of the covariance matrix (4). As stated by Machenko-Pastur theorem, almost surely


and , b= when . When , there is an additional Dirac measure at of mass . The is also named as Machenko-Pastur (M-P) Law[4]. Its shape is demonstrated in Fig. 1, which has parameter as and .

Fig. 1: MP law under , (a) and (b) .

Iii Sample covariance Matrix Analysis

Under hypothesis , the similar ESD analysis of the SCM has been proposed, such as in [8]. However, they did not provide the ESD when , and the method is quite explicit[3]. Here, we provide a theoretical result that the ESD of is the same as that of a central Wishart matrix, which has been studied extensively.

The received signal matrix is further decomposed as , where is the unknown signal matrix transmitted in , so


where , is a matrix with elements as in (2), has a size of , and the last part of is the correlation of white Gaussian noise.

The SCM in (6) is analyzed as




where is a random matrix with dimension . In (10), , and .

For the convenience of portraying the matrix entires, it is assumed that follows Binary distribution as

. Random variable

follows , because if and follows Binary distribution and standard complex Normal distribution,


Entries follows . Entries in have the same character.

If the signal follows Normal distribution,

converges to Normal distribution according to Law of Large Number. Simulations show that the Normal distributed signal has the same character as Binary distributed ones but with a slower convergence speed.

Lemma 1 (perturbation on SCM[5], Theorem 1.1)

For , , , , identically distributed, independent across for each , , as .
a) , and the distribution function of

converges almost surely in distribution to a probability distribution function (PDF)

as .
b) is Hermitian for which the ESD converges vaguely to almost surely, being a nonrandom distribution function.
c) , , and are independent.
Let . Then, almost surely, , the ESD of , converges vaguely, as , to a (non-random) distribution function , whose Stieltjes transform satisfies


According to lemma 1, the ESD of is determined by the ESD of and . Then an ideal matrix could be constructed, which has the similar ESD as (10) :

Lemma 2

When , for


where as a select matrix, , where is the last columns of . Then the ESD of (13) and (10) converge to the same PDF.

Proof III.1

Considering (13), we have




Thus the entries of and is the same. Obviously the ESD of and is the same, so the eigenvalue of and is the same. On the other hand, according to Theorem 1, the ESD of and converges to M-P law. Using Lemma 1, entries in and converges to the same PDF.

In lemma 2, an artificial is constructed whose ESD converges to the same distribution as . Though is an artificial matrix which is impossible to obtain, it could be substituted by central spiked Wishart matrices based on convergence in distribution, which is:

Theorem 1

The ESD of SCM (6) and a central Wishart matrix converge to the same PDF, where


and , , , , , .

Proof III.2

Let . The elements of the first columns are which follows .

For random matrix , the entires of first columns of follow the distribution , which is the same as .

The entries of and have the same distribution, and the ESD of both matrices fits theorem 2 of [5]. So the ESD of and converge to the same PDF. According to Lemma 2, the ESD of and converge to the same PDF.

is a central Wishart matrix, with the covariance matrix . This spike Wishart matrix has been widely studied[6], which could be used to design hypothesis detector in wireless communication.

Iv Application: Function of detection

According to Lemma 1, the signal detection in large array is translated to a standard high dimensional signal detection problem. Classical methods include detecting the ratio of biggest and smallest eigenvalue, detecting the trace of covariance matrix[9], and LRT[8]. Among them, LRT has a decent history and has been widely used.

According to chapter of [10], we assume that is the SCM formatted by (6), to test the hypothesis , where is the covariance matrix of a vector distributed according to . It is showed that the LRT


is unbiased LRT of (4). The detector is researched widely such as [6].

To make statistical integration about a parameter . It is natural to use the estimator


The log-liklihood ratio (LLR) could be further rewritten as


where is the -th eigenvalue of tested covariance matrix.

The ESD of has been rarely studied. The work [6] investigated the fluctuation of linear spectral statistics of form (17), with the form or and its results are based on , which is different from .

Under , it has


Substituting by , could fit the structure of in [6], and the results from [6] could be used directly. But in practice, the number of samples is larger than the antenna number in most scenario, which means is more practical.

Under and , equals and converges to normal distribution as shown in [6], which is




Under and , it has

: The ESD of and converge to the same PDF, where .

Proof IV.1

According to classical matrix theory, the non-zero eigenvalues of and are the same.

As introduced in [2]

, the outlier eigenvalues of

and could be elaborated by and .

For , according to Theorem 1.1 of [2], it has


For , using the same method as in (20), it is further written as


According to Theorem 1.2 in [2], the first outlier eigenvalues hold


According to the definition of and , we have


By calculation, it is easy to find that and converge to same limitation when , which is . The other eigenvalues of both matrices converge to Machenko-Pastur distribution, as in [2] and [6]. Thus, the ESD of and converge to the same distribution asymptotically.

Fig. 2: The eigenvalues density of when

Fig. 2 demonstrates the ESD density of . The largest eigenvalues are larger than . The rest follow M-P Law, which coincides with our analysis. This distribution is quite different with the eigenvalues of covariance matrix , which would stacked at and .

Combining Theorem 1 and Theorem 2, we have

: The ESD of SCM (6) and converge to the same distribution.

Based on [6], the LLR in (19) converges asymptotically to a Normal distribution with


Fig. 3 shows the distribution of under and , with . The simulated distributions look in good agreement with the theoretical distributions.

A classical signal detection algorithm is used (algorithm 1 in [9]) where we use the metric (19) as the core of algorithm.

Algorithm1: Signal detection using GLRT
1: At receiver, gets the received data
from each port synchronously and standardized respectively.
2: Organize the received signals together and form a matrix
as (5) and calculate the sample covariance matrix.
3: Compute using the eigenvalues of
the sample covariance matrix by (19).
4: Determine the probability of false alarm and
find out the threshold via computations.
5: if then
6: signal exists;
7: else
8: signal does not exist.
9: end if

Fig. 3: The (17) distributions with , simulations and theoretical, respectively.

As in literature [6], when and , the variance is the same as that of .

Interestingly, from (27), we find that the mean of is only related with and , and the variance is only related with no matter signal present or not.

The choice of threshold is a compromise between and . The probability of false alarm is


where the Q function is one minus the cumulative distribution function of the standardized normal random variable.

Under the limitation of the false alarming probability , the false alarm boundary is set as


Under , using as threshold, the distance between and follows Normal distribution as


the theoretical miss probability is


V Simulations

Fig. 4: The eigenvalues of , in Th. 1 and in Th. 2 when .

The first simulation is to test the theoretical results about (6), (16) and Theorem 2. A channel with 10 taps is used. We choose , , dB, respectively. Then from Fig. 4, it is obvious that the eigenvalues of (6), (14) and Theorem 2 coincide with each other, and the ESD of the three matrices are the same. In Fig. 4, the gaps between the largest eigenvalues of the three matrices are smaller than of the value. With the increment of and decrement of SNR, these gaps are smaller.

The second part of simulations is to detect LLR . In Fig. 5, we choose and , respectively. The is set as . Using (30), the theoretical miss probability is obtained. The algorithm aided by signal detection simulations is based on Algorithm 1 and (19). It is shown that, with the increment of sample number, the detection performance of the algorithm is improved. With the sample doubled, the detection threshold would reduce dB. In simulations, though the results of theoretical derivations and simulations are very close, the theoretical results look over-optimized the detection by smaller than 0.4

. Fig. 6 is the ROC (receiver operating characteristic) curve with

dB and dB. In Fig. 6, the over-optimization is also found, where the ROC in theoretical is slightly better than empirical.

The gaps in Fig. 4, 5 and 6 are brought by the infinite approximation of , as well as the approximation character of variance estimation which is referred from [6]. Reviewing Fig. 3, the variance under looks larger than under , which makes the empirical density of higher than but the theoretical variance is still the same. So even the variance analysis of (24) in [6] is the most accurate work till now, there still has room to improve.

Fig. 5: Miss probability under different antenna number with different SNR

Fig. 6: ROC curve under and .

Vi Conclusion

In this paper, we have proposed a detection algorithm in the large antenna array system when the antenna number and the samples number are comparable. Two theorems have been presented to connect the detection with the newest statistical results. Furthermore, we have given a new detection metric which is based on an asymptotic result in RMT. Simulation results have shown that the model agrees with real simulations very well. In further works, we would simplify the complexity using distributed algorithm. This paper has given results in asymptotic scenario. It is interesting to investigate the bound when and are not so large. Furthermore, this results could be extended to other applications easily, such as a multiuser with Rician channel scenario.


  • [1] T. T. Cai, C. H. Zhang, and H. H. Zhou, “Optimal rates of convergence for covariance matrix estimation”, The Annals of Statistics, vol. 38, no. 4, pp. 2118-2144, 2010.
  • [2] Baik. Jinho, and J. W. Silverstein, “Eigenvalues of large sample covariance matrices of spiked population models,”

    Journal of Multivariate Analysis

    vol. 97, no. 6, pp.1382-1408, 2006.
  • [3] Marwa Banna, Jamal Najim, Jianfeng Yao, “A CLT for linear spectral statistics of large random information-plus-noise matrices”, Pre-print 2018.
  • [4] Marchenko V A, Pastur L A, “Distribution of eigenvalues for some sets of random matrices”, Mathemat ics of the USSR-Sbornik, vol. 1, no. 1, pp. 507-536, 1967.
  • [5] Silverstein. Jack W, “The Stieltjes transform and its role in eigenvalue behavior of large dimensional random matrices.” Random Matrix Theory and Its Applications, pp.1-25, 2009.
  • [6] Wang. Qinwen, Silverstein, Jack W., Yao. Jian-feng, “A note on the CLT of the LSS for sample covariance matrix from a spiked population model”, Journal of Multivariate Analysis, vol. 130, pp. 194-207, Sep. 2014.
  • [7] Guanping Lu, J. Wu, and R. Ying, “Filtered multitone transmission with variable subcarrier bandwidths,” IEEE International Conference on Communication Workshop IEEE, 2015.
  • [8] Bianchi, Debbah, Maida, Najim, “Performance of statistical tests for single source detection using random matrix theory”, IEEE Transactions on Information Theory, vol. 57, no. 4, 2400-2419, 2010.
  • [9] Lin, Feng, Robert Qiu, et al., “Generalized fmd detection for spectrum sensing under low signal-to-noise ratio”, Communications Letters, IEEE, vol. 16, No .5, pp. 604-607, May 2012.
  • [10] Anderson, Theodore Wilbur, et al., An introduction to multivariate statistical analysis, New York: Wiley, 1958.
  • [11] A. M. Tulino, S. Verdu, Random Matrix Theory and Wireless Communications, now Publishers, 2004.