All-or-Nothing Phenomena: From Single-Letter to High Dimensions
share
We consider the linear regression problem of estimating a p-dimensional vector β from n observations Y = X β + W, where β_j i.i.d.∼π for a real-valued distribution π with zero mean and unit variance, X_iji.i.d.∼N(0,1), and W_ii.i.d.∼N(0, σ^2). In the asymptotic regime where n/p →δ and p/ σ^2 →snr for two fixed constants δ, snr∈ (0, ∞) as p →∞, the limiting (normalized) minimum mean-squared error (MMSE) has been characterized by the MMSE of an associated single-letter (additive Gaussian scalar) channel. In this paper, we show that if the MMSE function of the single-letter channel converges to a step function, then the limiting MMSE of estimating β in the linear regression problem converges to a step function which jumps from 1 to 0 at a critical threshold. Moreover, we establish that the limiting mean-squared error of the (MSE-optimal) approximate message passing algorithm also converges to a step function with a larger threshold, providing evidence for the presence of a computational-statistical gap between the two thresholds.
READ FULL TEXT