Phase transition and higher order analysis of L_q regularization under dependence
share
We study the problem of estimating a k-sparse signal _0∈ R^p from a set of noisy observations y∈ R^n under the model y= X+ w, where X∈ R^n× p is the measurement matrix the row of which is drawn from distribution N(0,). We consider the class of L_q-regularized least squares (LQLS) given by the formulation (λ,q)=argmin_∈ R^p1/2 y- X^2_2+λ_q^q, where ·_q (0≤ q≤ 2) denotes the L_q-norm. In the setting p,n,k→∞ with fixed k/p=ϵ and n/p=δ, we derive the asymptotic risk of (λ,q) for arbitrary covariance matrix which generalizes the existing results for standard Gaussian design, i.e. X_iji.i.d∼N(0,1). We perform a higher-order analysis for LQLS in the small-error regime in which the first dominant term can be used to determine the phase transition behavior of LQLS. Our results show that the first dominant term does not depend on the covariance structure of in the cases 0≤ q< 1 and 1< q≤ 2 which indicates that the correlations among predictors only affect the phase transition curve in the case q=1 a.k.a. LASSO. To study the influence of the covariance structure of on the performance of LQLS in the cases 0≤ q< 1 and 1<q≤ 2, we derive the explicit formulas for the second dominant term in the expansion of the asymptotic risk in terms of small error. Extensive computational experiments confirm that our analytical predictions are consistent with numerical results.
READ FULL TEXT