Statistical-Computational Trade-offs in Tensor PCA and Related Problems via Communication Complexity

04/15/2022
by   Rishabh Dudeja, et al.
0

Tensor PCA is a stylized statistical inference problem introduced by Montanari and Richard to study the computational difficulty of estimating an unknown parameter from higher-order moment tensors. Unlike its matrix counterpart, Tensor PCA exhibits a statistical-computational gap, i.e., a sample size regime where the problem is information-theoretically solvable but conjectured to be computationally hard. This paper derives computational lower bounds on the run-time of memory bounded algorithms for Tensor PCA using communication complexity. These lower bounds specify a trade-off among the number of passes through the data sample, the sample size, and the memory required by any algorithm that successfully solves Tensor PCA. While the lower bounds do not rule out polynomial-time algorithms, they do imply that many commonly-used algorithms, such as gradient descent and power method, must have a higher iteration count when the sample size is not large enough. Similar lower bounds are obtained for Non-Gaussian Component Analysis, a family of statistical estimation problems in which low-order moment tensors carry no information about the unknown parameter. Finally, stronger lower bounds are obtained for an asymmetric variant of Tensor PCA and related statistical estimation problems. These results explain why many estimators for these problems use a memory state that is significantly larger than the effective dimensionality of the parameter of interest.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/10/2020

Statistical Query Lower Bounds for Tensor PCA

In the Tensor PCA problem introduced by Richard and Montanari (2014), on...
research
05/21/2020

Computationally efficient sparse clustering

We study statistical and computational limits of clustering when the mea...
research
12/29/2020

Inference for Low-rank Tensors – No Need to Debias

In this paper, we consider the statistical inference for several low-ran...
research
12/23/2016

Spectral algorithms for tensor completion

In the tensor completion problem, one seeks to estimate a low-rank tenso...
research
06/17/2022

Tensor-on-Tensor Regression: Riemannian Optimization, Over-parameterization, Statistical-computational Gap, and Their Interplay

We study the tensor-on-tensor regression, where the goal is to connect t...
research
05/22/2018

Solvable Integration Problems and Optimal Sample Size Selection

We want to compute the integral of a function or the expectation of a ra...
research
06/18/2020

Free Energy Wells and Overlap Gap Property in Sparse PCA

We study a variant of the sparse PCA (principal component analysis) prob...

Please sign up or login with your details

Forgot password? Click here to reset