Coherence Statistics of Structured Random Ensembles and Support Detection Bounds for OMP

A structured random matrix ensemble that maintains constant modulus entries and unit-norm columns, often called a random phase-rotated (RPR) matrix, is considered in this paper. We analyze the coherence statistics of RPR measurement matrices and apply them to acquire probabilistic performance guarantees of orthogonal matching pursuit (OMP) for support detection (SD). It is revealed via numerical simulations that the SD performance guarantee provides a tight characterization, especially when the signal is sparse.

Authors

• 3 publications
• 11 publications
• 3 publications
• 2 publications
• An Optimal Restricted Isometry Condition for Exact Sparse Recovery with Orthogonal Least Squares

The orthogonal least squares (OLS) algorithm is popularly used in sparse...
10/24/2019 ∙ by Junhan Kim, et al. ∙ 0

• A Quasi-Orthogonal Matching Pursuit Algorithm for Compressive Sensing

In this paper, we propose a new orthogonal matching pursuit algorithm ca...
07/18/2020 ∙ by Ming-Jun Lai, et al. ∙ 0

• Signal and Noise Statistics Oblivious Orthogonal Matching Pursuit

Orthogonal matching pursuit (OMP) is a widely used algorithm for recover...
06/02/2018 ∙ by Sreejith Kallummil, et al. ∙ 0

• Convex Reconstruction of Structured Matrix Signals from Random Linear Measurements (I): Theoretical Results

We investigate the problem of reconstructing n-by-n column-wise sparse m...
10/19/2019 ∙ by Yuan Tian, et al. ∙ 0

• Support Recovery for Orthogonal Matching Pursuit: Upper and Lower bounds

This paper studies the problem of sparse regression where the goal is to...
06/27/2021 ∙ by Raghav Somani, et al. ∙ 0

• Householder Dice: A Matrix-Free Algorithm for Simulating Dynamics on Gaussian and Random Orthogonal Ensembles

This paper proposes a new algorithm, named Householder Dice (HD), for si...
01/19/2021 ∙ by Yue M. Lu, et al. ∙ 0

• On the similarity between Nakagami-m Fading distribution and the Gaussian ensembles of random matrix theory

We report the similarity between the Nakagami-m fading distribution and ...
03/23/2018 ∙ by Sherif M. Abuelenin, et al. ∙ 0

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Random matrix ensembles have found wide applications in fields of wireless communications and signal processing [1, 2, 3, 4]. Despite the fact that most studied Gaussian measurement ensembles offer trackable analyses and appealing results [5, 6, 7], they are of somewhat limited use in practical applications because the design of measurement matrices is usually subject to physical or other constraints provided by a specific system architecture. It is desirable to explore random matrix ensembles with hidden structure from a computational and an application-oriented point of view.

Coherence has been utilized to measure the quality of the measurement matrix [8]

. Analysis of coherence statistics of random vectors/matrices plays an important role in solving a series of signal processing problems including the Grassmannian line packing

[9, 10], random vector quantization [11, 12], and support detection (SD) [5, 13, 14, 15]. In particular, the performance of SD considerably varies with the characteristics of measurement matrices. There is a certain class of random matrix ensembles with hidden structures that can demonstrate an improvement in SD performance guarantees compared to Gaussian ensembles [5]. Distinguished from the Gaussian measurement matrix that does not contain hidden constraints, the random phase-rotated (RPR) measurement matrix, where each entry is drawn from the constant modulus uniform phase rotation distribution, brings the benefits of maintaining unit-norm columns and constant modulus entries of the measurement matrix. This measurement ensemble has been utilized in advanced beamforming and precoding for wireless communications [16, 17].

In this paper, we calculate high probability bounds on the coherence statistics of RPR measurement matrices and apply them to obtain SD performance guarantees for orthogonal matching pursuit (OMP), which is a low-complexity, greedy approach for SD

[18, 5]. The performance bound is in terms of the required number of measurements for any given number of supports and system dimensions. A free variable is introduced, which is optimized to further tighten the performance bound. The main motivation is that previous work relying on the coherence property did not contain hidden constraints that are suitable for SD of OMP. Numerical evaluations demonstrate that the analyzed SD performance guarantee of OMP is tight, especially when the signal is sparse.

Ii Coherence Statistics

Suppose a random measurement matrix with being the th column of . Each entry of is constant modulus and drawn from the random phase rotation variable as

 Amn=1√MejΘmn, (1)

where denotes the th row and th column entry of , , , and the phase

is an independent and identically distributed (i.i.d.) uniform random variable, i.e.,

. With the construction in (1), maintains , .

The coherence of is the maximum absolute correlation between two distinct columns of [19], which is given by

 μ(A)≜maxi≠j|a∗iaj|, (2)

where denotes the conjugate transpose. Characterizing the distribution of is of interest - however, it is challenging to directly derive the distribution of when follows (1

). To circumvent this difficulty, we relegate to find a lower bound on the cumulative distribution function (CDF) of

instead. We start by building a connection between the vector drawn from the distribution in (1) and the vector consisting of Bernoulli random variables.

Lemma 1.

Let and be random vectors with i.i.d. entries , , and with equal probability for , respectively. Then, for any unit-norm vector , the following inequality holds

 E[|p∗u|2k]≤E[|q∗¯u|2k], (3)

where has each entry , , is a nonnegative integer, and the expectations are taken over and , respectively.

Proof: See Appendix A.

Based on Lemma 1, we characterize a bound on the distribution of below.

Lemma 2.

Suppose the vectors and defined in Lemma 1. Then, for any , the following inequality holds

 Pr(|p∗u|≥δ)≤(1−2g)−12e−δ2Mg, g>2. (4)

Proof: See Appendix B.

Remark 1.

It is also possible to derive an upper bound on by leveraging the matrix Bernstein inequality [20, Theorem 1.6.2], which leads to . However, this bound is looser than that in (4).

A lower bound on the CDF of in (2) can be found.

Theorem 1.

Suppose a matrix consisting of i.i.d. entries , , , . Then, the following holds for ,

 Pr(μ(A)<δ)≥(1−(1−2g)−12e−δ2Mg)N(N−1)2. (5)
Proof.

The inner product between two distinct column vectors of satisfies

 a∗n1an2=M∑m=11MejΔΘmd=M∑m=11Mejξm=p∗¯u, (6)

where ,

, is the difference between two independent uniform random variables, whose probability density function is given by

 p(ΔΘm)={2π−|ΔΘm|4π2,if −2π≤ΔΘm<2π0,otherwise.

In (6), follows the same definition in Lemma 1, and we use the fact that and , in which is the modulo of . Note that and it verifies that has the same distribution as in (6), where is the equality in distribution.

By Lemma 2, we now have . Then, the maximum order statistic of is lower bounded by

 = Pr(μ(A)≤δ) ≥ (1−(1−2g)−12e−δ2Mg)N(N−1)2.

This completes the proof. ∎

Remark 2.

Because Bernoulli random matrices with each entry filled with can be regarded as a special case of the RPR matrices in (1) when with equal probability, , the coherence statistic in (5) also holds for the Bernoulli random matrix.

Iii Support Detection Bounds for OMP

In this section, the coherence statistics of RPR measurement matrices are applied to obtain the probability bounds of SD for OMP.

Iii-a Measurement Model and OMP Algorithm

Suppose a measurement model

 y=Ax, (7)

where each entry of follows (1). Here, the assumption is that the number of measurements is smaller than the signal dimension , i.e., . The signal in (7) has nonzero elements (supports) whose indexes are defined by the support set

 S=supp(x)={n1,…,nK|xnk≠0,nk∈{1,…,N}}, (8)

where . The goal is to detect the support set from the measurement in (7).

An iterative procedure of OMP for SD is depicted in Algorithm 1 for the measurement model in (7). To make sure that the active index determined in Step 4 is a true support, the following sufficient condition [19] should be met,

 (9)

where is the submatrix formed by taking the columns of indexed by and is the complementary submatrix of . The nonzero coefficients estimated in Step 7 are formed by extracting the nonzero elements of indexed by and given by . It is crucial to recognize that the updated residual is orthogonal to the columns of . The OMP detects one support at each iteration and runs for exactly iterations.

Iii-B Support Detection Performance Guarantee

We provide the SD performance guarantee of the OMP in Algorithm 1 as follows.

Theorem 2.

Suppose the measurement model in (7) with the RPR measurement matrix based on (1). Then, the OMP in Algorithm 1 detects the supports of for any with

 Pr(VSSD)≥1−(1−2g)−12KN⋅e−MgK2, g>2, (10)

where is the event of successful SD (SSD) after iterations. When the number of measurements satisfies

 M≥gK2ln(KNϵ√1−2g), g>2, (11)

for , Algorithm 1 satisfies .

Proof: See Appendix C.

To further tighten the lower bound in (11), we optimize the free variable by minimizing the right hand side (r.h.s.) of (11) such that

 gopt=argming>2f(g)≜gK2ln(KNϵ√1−2g). (12)
Theorem 3.

The objective function in (12) is convex for and a closed-form expression of is given by

 gopt=21+(W−1(−(ϵKN)2e−1))−1, (13)

where is the lower branch of the Lambert function [21], defined by for .

Proof: See Appendix D.

Iv Numerical Simulations

To verify the SD performance guarantee in (11), we perform Monte Carlo simulations in Fig. 1, where the probability of SD error, i.e., , across different numbers of measurements for and , is evaluated. In the simulation, the signal is generated by randomly choosing supports with each support having , for , and we compare with the existing coherence-based SD performance guarantee for the Gaussian random measurement matrix [5]. In Fig. 1, the vertical lines denote the minimum required to guarantee the SD error rate , where these values are given by the r.h.s. of (11) for the RPR measurements , and for the Gaussian case [5], respectively. Seen from Fig. 1, the obtained SD performance guarantee of RPR matrices provides a tighter characterization than the Gaussian case when the signal is sparse, i.e., is small.

V Conclusion and Discussion

The coherence statistics of RPR matrices were analyzed and applied to obtain the SD performance guarantees of OMP. The introduced free variable was optimized to further tighten the SD bound. Numerical simulations corroborated the theoretical findings and revealed that including the constant modulus and unit-norm structure for random measurement ensembles is desirable for SD using OMP.

In this work, we focused on the coherence statistics of RPR matrices to show the SD performance guarantees of OMP. In particular, we proved that OMP can achieve SSD with high probability, provided RPR measurements. It is of interest to compare our coherence-based analysis with the restricted isometry property (RIP)-based result since they are two main techniques in analyzing the performance guarantees of SD for OMP. By using the concentration inequality in [4, Theorem 2] and the method of proving the RIP for random matrices in [22, Theorem 5.2], one can obtain that is sufficient for the RPR matrices to satisfy the RIP with high probability, where is the restricted isometry constant. With being a strict condition of SSD for OMP [23], the RIP-based SD bound can be given by , which is on par with our coherence-based results in Theorem 2.

Finally, one limitation of the work is that the SD bound becomes loose as grows. Seen from Fig. 1, there is still room for further improvement by investigating a new structure of random measurement ensembles, which is subject to future research.

Appendix A Proof of Lemma 1

Proof.

The left hand side (l.h.s.) and r.h.s. of (3) can be rewritten as and , respectively, where , , with equal probability. Thus, showing the inequality in (3) is equivalent to showing

 MkE[∣∣M∑m=1ume−jθm∣∣2k]≤E[∣∣M∑m=1ζm∣∣2k]. (14)

The l.h.s. of (14) can be simplified as

 MkE[∣∣M∑m=1ume−jθm∣∣2k] (a)=MkE[(∣∣M∑m=1umcos(θm)∣∣2+∣∣M∑m=1umsin(θm)∣∣2)k] (b)=MkE[(M∑m=1|um|2)k](c)=Mk, (15)

where follows from the equality , is due to the fact that for , and holds because . Expanding the r.h.s. of (14) leads to

 E[∣∣M∑m=1ζm∣∣2k] = E[(M+G(M))k] (16) = E[k∑i=0(ki)Mk−iG(M)i]≥Mk,

where . The inequality in (16) becomes the equality only if because for . On the other hand, when , the strict inequality in (16) holds because for any positive integers , , leading to for . Combining (15) and (16) results in (14). ∎

Appendix B Proof of Lemma 2

Proof.

By using Markov’s inequality, we have for ,

 Pr(|p∗u|≥δ)=Pr(|p∗u|2≥δ2)≤E[eh|p∗u|2]e−hδ2. (17)

The term in (17) can further be upper bounded for by

 E[eh|p∗u|2]≤E[eh|q∗¯u|2]≤(1−2hM)−12, (18)

where the first inequality is due to the Taylor series expansion of and Lemma 1 applied to , and follows the same definition in Lemma 1. The last step in (18) follows from the inequality for in [24, Lemma 5.2].

Inserting (18) into (17) leads to

 Pr(|p∗u|≥δ)≤(1−2hM)−12e−hδ2. (19)

Because the inequality holds for any , substituting , , into (19) completes the proof. ∎

Appendix C Proof of Theorem 2

Proof.

The proof is inspired by a similar theorem in [5, Theorem 6] and refines the results for the RPR measurement ensembles in conjunction with Lemma 2 and Theorem 1. We first elaborate two events: 1) is defined on the basis of the condition in (9) as ; and 2) The event that is bounded by , i.e., . The event is to restrict the on a special class of to ease the bound analysis below.

Conditioned on the event , the probability of SSD can be lower bounded by

 Pr(VSSD)≥Pr(VSSD∩D)=Pr(VSSD|D)Pr(D). (20)

From Theorem 1, in (20) can be lower bounded by

 Pr(D) = Pr(μ(AS)≤1K) (21) ≥ [1−(1−2g)−12e−MgK2]K(K−1)2,

where . The conditional probability on the r.h.s. of (20) can be lower bounded by

 Pr(VSSD|D) = Pr(maxt∥A∗SCrt−1∥∞∥A∗Srt−1∥∞<1∣∣D) (22) (a)≥ Pr(maxt√Kmaxj∈SC|a∗jrt−1|∥A∗Srt−1∥2<1∣∣D) (b)≥ (c)= ∏j∈SCPr(maxt |a∗jbt−1|<1K∣∣D) (d)≥ [1−(1−2g)−12e−MgK2]K(N−K)

where is due to the inequality for , comes from where and because by applying Gershgorin disc theorem [25], holds due to the fact that the columns of are independent, and is due to Lemma 2.

Substitute (21) and (22) into (20) yields

 Pr(VSSD) ≥ [1−(1−2g)−12e−MgK2]K(N−K)+K(K−1)2 (a)≥ 1−(1−2g)−12[K(N−K)+K(K−1)2]e−MgK2 ≥ 1−(1−2g)−12KNe−MgK2,

where holds because and . Setting and taking the natural logarithm of both sides reveals that when . ∎

Appendix D Proof of Theorem 3

Proof.

We first claim that the objective function in (12) is convex for . To show this, we check the second-order condition , where is the second-order derivative of . After some algebraic manipulations, the first and second-order derivatives of can be written, respectively, as and . Because for , is convex.

The optimality condition of (12) can now be described by using the first-order optimality condition as

 f(gopt)=goptgopt−2K2. (23)

Let , equivalently . Then, by (12), the equality in (23) can be rewritten as . This yields , which follows from the definition of the lower branch of the Lambert function and [21]. Now, by the equality , we finally have (13). This completes the proof. ∎

References

• [1] Y. Liang, G. Pan, and Z. D. Bai, “Asymptotic performance of mmse receivers for large systems using random matrix theory,” IEEE Transactions on Information Theory, vol. 53, no. 11, pp. 4173–4190, Nov 2007.
• [2]

R. Menon, P. Gerstoft, and W. S. Hodgkiss, “Asymptotic eigenvalue density of noise covariance matrices,”

IEEE Transactions on Signal Processing, vol. 60, no. 7, pp. 3415–3424, July 2012.
• [3] K. Elkhalil, A. Kammoun, T. Y. Al-Naffouri, and M. Alouini, “Measurement selection: A random matrix theory approach,” IEEE Transactions on Wireless Communications, vol. 17, no. 7, pp. 4899–4911, July 2018.
• [4] W. Zhang, T. Kim, D. J. Love, and E. Perrins, “Leveraging the restricted isometry property: Improved low-rank subspace decomposition for hybrid millimeter-wave systems,” IEEE Transactions on Communications, vol. 66, no. 11, pp. 5814–5827, Nov 2018.
• [5] J. A. Tropp and A. C. Gilbert, “Signal recovery from random measurements via orthogonal matching pursuit,” IEEE Transactions on Information Theory, vol. 53, no. 12, pp. 4655–4666, Dec 2007.
• [6] A. K. Fletcher and S. Rangan, “Orthogonal matching pursuit: A brownian motion analysis,” IEEE Transactions on Signal Processing, vol. 60, no. 3, pp. 1010–1021, March 2012.
• [7] N. Lee, “Map support detection for greedy sparse signal recovery algorithms in compressive sensing,” IEEE Transactions on Signal Processing, vol. 64, no. 19, pp. 4987–4999, Oct 2016.
• [8] D. L. Donoho and X. Huo, “Uncertainty principles and ideal atomic decomposition,” IEEE Transactions on Information Theory, vol. 47, no. 7, pp. 2845–2862, Nov 2001.
• [9] D. J. Love, R. W. Heath, and T. Strohmer, “Grassmannian beamforming for multiple-input multiple-output wireless systems,” IEEE Transactions on Information Theory, vol. 49, no. 10, pp. 2735–2747, Oct 2003.
• [10] K. K. Mukkavilli, A. Sabharwal, E. Erkip, and B. Aazhang, “On beamforming with finite rate feedback in multiple-antenna systems,” IEEE Transactions on Information Theory, vol. 49, no. 10, pp. 2562–2579, Oct 2003.
• [11] N. Jindal, “Mimo broadcast channels with finite-rate feedback,” IEEE Transactions on Information Theory, vol. 52, no. 11, pp. 5045–5060, Nov 2006.
• [12] C. K. Au-yeung and D. J. Love, “On the performance of random vector quantization limited feedback beamforming in a miso system,” IEEE Transactions on Wireless Communications, vol. 6, no. 2, pp. 458–462, Feb 2007.
• [13] Z. Ben-Haim, Y. C. Eldar, and M. Elad, “Coherence-based performance guarantees for estimating a sparse vector under random noise,” IEEE Transactions on Signal Processing, vol. 58, no. 10, pp. 5030–5043, Oct 2010.
• [14] A. Bracher, G. Pope, and C. Studer, “Coherence-based probabilistic recovery guarantees for sparsely corrupted signals,” in 2012 IEEE Information Theory Workshop, Sep. 2012, pp. 307–311.
• [15] E. Malhotra, K. Gurumoorthy, and A. Rajwade, “Stronger recovery guarantees for sparse signals exploiting coherence structure in dictionaries,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), March 2017, pp. 6085–6089.
• [16] S. Hur, T. Kim, D. J. Love, J. V. Krogmeier, T. A. Thomas, and A. Ghosh, “Millimeter wave beamforming for wireless backhaul and access in small cell networks,” IEEE Transactions on Communications, vol. 61, no. 10, pp. 4391–4403, Oct 2013.
• [17] T. Kim and D. J. Love, “Virtual AoA and AoD estimation for sparse millimeter wave MIMO channels,” in 2015 IEEE 16th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), June 2015, pp. 146–150.
• [18] T. T. Cai and L. Wang, “Orthogonal matching pursuit for sparse signal recovery with noise,” IEEE Transactions on Information Theory, vol. 57, no. 7, pp. 4680–4688, July 2011.
• [19] J. A. Tropp, “Greed is good: algorithmic results for sparse approximation,” IEEE Transactions on Information Theory, vol. 50, no. 10, pp. 2231–2242, Oct 2004.
• [20] J. A. Tropp, “An introduction to matrix concentration inequalities,”

Foundations and Trends® in Machine Learning

, vol. 8, no. 1-2, pp. 1–230, 2015.
• [21] I. S. Gradshteyn and I. M. Ryzhik, Table of integrals, series, and products, 7th ed.   Elsevier/Academic Press, Amsterdam, 2007.
• [22] R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin, “A simple proof of the restricted isometry property for random matrices,” Constructive Approximation, vol. 28, no. 3, pp. 253–263, Dec 2008.
• [23] M. A. Davenport and M. B. Wakin, “Analysis of orthogonal matching pursuit using the restricted isometry property,” IEEE Transactions on Information Theory, vol. 56, no. 9, pp. 4395–4401, Sept 2010.
• [24] D. Achlioptas, “Database-friendly random projections: Johnson-lindenstrauss with binary coins,” J. Comput. Syst. Sci., vol. 66, no. 4, pp. 671–687, June 2003.
• [25] G. H. Golub and C. F. V. Loan, Matrix Computations (3rd Ed.).   Baltimore, MD, USA: Johns Hopkins University Press, 1996.