The All-or-Nothing Phenomenon in Sparse Tensor PCA

07/22/2020
by   Jonathan Niles-Weed, et al.
0

We study the statistical problem of estimating a rank-one sparse tensor corrupted by additive Gaussian noise, a model also known as sparse tensor PCA. We show that for Bernoulli and Bernoulli-Rademacher distributed signals and for all sparsity levels which are sublinear in the dimension of the signal, the sparse tensor PCA model exhibits a phase transition called the all-or-nothing phenomenon. This is the property that for some signal-to-noise ratio (SNR) SNR_c and any fixed ϵ>0, if the SNR of the model is below (1-ϵ)SNR_c, then it is impossible to achieve any arbitrarily small constant correlation with the hidden signal, while if the SNR is above (1+ϵ)SNR_c, then it is possible to achieve almost perfect correlation with the hidden signal. The all-or-nothing phenomenon was initially established in the context of sparse linear regression, and over the last year also in the context of sparse 2-tensor (matrix) PCA, Bernoulli group testing, and generalized linear models. Our results follow from a more general result showing that for any Gaussian additive model with a discrete uniform prior, the all-or-nothing phenomenon follows as a direct outcome of an appropriately defined "near-orthogonality" property of the support of the prior distribution.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/11/2021

The Complexity of Sparse Tensor PCA

We study the problem of sparse tensor principal component analysis: give...
research
02/24/2021

It was "all" for "nothing": sharp phase transitions for noiseless discrete channels

We establish a phase transition known as the "all-or-nothing" phenomenon...
research
02/05/2021

Sparse Normal Means Estimation with Sublinear Communication

We consider the problem of sparse normal means estimation in a distribut...
research
03/03/2023

Statistical-Computational Tradeoffs in Mixed Sparse Linear Regression

We consider the problem of mixed sparse linear regression with two compo...
research
12/22/2016

Statistical limits of spiked tensor models

We study the statistical limits of both detecting and estimating a rank-...
research
11/30/2017

Phase Transitions in Approximate Ranking

We study the problem of approximate ranking from observations of pairwis...
research
05/04/2018

Global testing under the sparse alternatives for single index models

For the single index model y=f(β^τx,ϵ) with Gaussian design, and β is a...

Please sign up or login with your details

Forgot password? Click here to reset