DeepAI AI Chat
Log In Sign Up

Asymptotic errors for convex penalized linear regression beyond Gaussian matrices

by   Cédric Gerbelot, et al.

We consider the problem of learning a coefficient vector x_0 in R^N from noisy linear observations y=Fx_0+w in R^M in the high dimensional limit M,N to infinity with α=M/N fixed. We provide a rigorous derivation of an explicit formula – first conjectured using heuristic methods from statistical physics – for the asymptotic mean squared error obtained by penalized convex regression estimators such as the LASSO or the elastic net, for a class of very generic random matrices corresponding to rotationally invariant data matrices with arbitrary spectrum. The proof is based on a convergence analysis of an oracle version of vector approximate message-passing (oracle-VAMP) and on the properties of its state evolution equations. Our method leverages on and highlights the link between vector approximate message-passing, Douglas-Rachford splitting and proximal descent algorithms, extending previous results obtained with i.i.d. matrices for a large class of problems. We illustrate our results on some concrete examples and show that even though they are asymptotic, our predictions agree remarkably well with numerics even for very moderate sizes.


page 1

page 2

page 3

page 4


Robustness in sparse linear models: relative efficiency based on robust approximate message passing

Understanding efficiency in high dimensional linear models is a longstan...

Efficient Designs of SLOPE Penalty Sequences in Finite Dimension

In linear regression, SLOPE is a new convex analysis method that general...

Proximal Estimation and Inference

We build a unifying convex analysis framework characterizing the statist...

Spectral Universality of Regularized Linear Regression with Nearly Deterministic Sensing Matrices

It has been observed that the performances of many high-dimensional esti...

High-temperature Expansions and Message Passing Algorithms

Improved mean-field technics are a central theme of statistical physics ...

RLS-Based Detection for Massive Spatial Modulation MIMO

Most detection algorithms in spatial modulation (SM) are formulated as l...