Tight Query Complexity Lower Bounds for PCA via Finite Sample Deformed Wigner Law

04/04/2018
by   Max Simchowitz, et al.
0

We prove a query complexity lower bound for approximating the top r dimensional eigenspace of a matrix. We consider an oracle model where, given a symmetric matrix M∈R^d × d, an algorithm Alg is allowed to make T exact queries of the form w^(i) = Mv^(i) for i in {1,...,T}, where v^(i) is drawn from a distribution which depends arbitrarily on the past queries and measurements {v^(j),w^(i)}_1 < j < i-1. We show that for every gap∈ (0,1/2], there exists a distribution over matrices M for which 1) gap_r(M) = Ω(gap) (where gap_r(M) is the normalized gap between the r and r+1-st largest-magnitude eigenvector of M), and 2) any algorithm Alg which takes fewer than const×r d/√(gap) queries fails (with overwhelming probability) to identity a matrix V∈R^d × r with orthonormal columns for which 〈V, MV〉> (1 - const×gap)∑_i=1^r λ_i(M). Our bound requires only that d is a small polynomial in 1/gap and r, and matches the upper bounds of Musco and Musco '15. Moreover, it establishes a strict separation between convex optimization and randomized, "strict-saddle" non-convex optimization of which PCA is a canonical example: in the former, first-order methods can have dimension-free iteration complexity, whereas in PCA, the iteration complexity of gradient-based methods must necessarily grow with the dimension.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/14/2017

On the Gap Between Strict-Saddles and True Convexity: An Omega(log d) Lower Bound for Eigenvector Approximation

We prove a query complexity lower bound on rank-one principal component ...
research
02/09/2023

Quadratic Memory is Necessary for Optimal Query Complexity in Convex Optimization: Center-of-Mass is Pareto-Optimal

We give query complexity lower bounds for convex optimization and the re...
research
04/25/2023

Towards Characterizing the First-order Query Complexity of Learning (Approximate) Nash Equilibria in Zero-sum Matrix Games

In the first-order query model for zero-sum K× K matrix games, playersob...
research
06/05/2023

Curvature and complexity: Better lower bounds for geodesically convex optimization

We study the query complexity of geodesically convex (g-convex) optimiza...
research
03/08/2021

On the Oracle Complexity of Higher-Order Smooth Non-Convex Finite-Sum Optimization

We prove lower bounds for higher-order methods in smooth non-convex fini...
research
10/05/2022

Fisher information lower bounds for sampling

We prove two lower bounds for the complexity of non-log-concave sampling...
research
10/18/2018

Testing Matrix Rank, Optimally

We show that for the problem of testing if a matrix A ∈ F^n × n has rank...

Please sign up or login with your details

Forgot password? Click here to reset