DeepAI AI Chat
Log In Sign Up

The All-or-Nothing Phenomenon in Sparse Tensor PCA

by   Jonathan Niles-Weed, et al.
NYU college

We study the statistical problem of estimating a rank-one sparse tensor corrupted by additive Gaussian noise, a model also known as sparse tensor PCA. We show that for Bernoulli and Bernoulli-Rademacher distributed signals and for all sparsity levels which are sublinear in the dimension of the signal, the sparse tensor PCA model exhibits a phase transition called the all-or-nothing phenomenon. This is the property that for some signal-to-noise ratio (SNR) SNR_c and any fixed ϵ>0, if the SNR of the model is below (1-ϵ)SNR_c, then it is impossible to achieve any arbitrarily small constant correlation with the hidden signal, while if the SNR is above (1+ϵ)SNR_c, then it is possible to achieve almost perfect correlation with the hidden signal. The all-or-nothing phenomenon was initially established in the context of sparse linear regression, and over the last year also in the context of sparse 2-tensor (matrix) PCA, Bernoulli group testing, and generalized linear models. Our results follow from a more general result showing that for any Gaussian additive model with a discrete uniform prior, the all-or-nothing phenomenon follows as a direct outcome of an appropriately defined "near-orthogonality" property of the support of the prior distribution.


page 1

page 2

page 3

page 4


The Complexity of Sparse Tensor PCA

We study the problem of sparse tensor principal component analysis: give...

It was "all" for "nothing": sharp phase transitions for noiseless discrete channels

We establish a phase transition known as the "all-or-nothing" phenomenon...

Sparse Normal Means Estimation with Sublinear Communication

We consider the problem of sparse normal means estimation in a distribut...

Statistical-Computational Tradeoffs in Mixed Sparse Linear Regression

We consider the problem of mixed sparse linear regression with two compo...

Statistical limits of spiked tensor models

We study the statistical limits of both detecting and estimating a rank-...

Phase Transitions in Approximate Ranking

We study the problem of approximate ranking from observations of pairwis...

Global testing under the sparse alternatives for single index models

For the single index model y=f(β^τx,ϵ) with Gaussian design, and β is a...