Exact spectral norm error of sample covariance
share
Let X_1,…,X_n be i.i.d. centered Gaussian vectors in ℝ^p with covariance Σ, and let Σ̂≡ n^-1∑_i=1^n X_iX_i^⊤ be the sample covariance. A central object of interest in the non-asymptotic theory of sample covariance is the spectral norm error ||Σ̂-Σ|| of the sample covariance Σ̂. In the path-breaking work of Koltchinskii and Lounici [KL17a], the `zeroth-order' magnitude of ||Σ̂-Σ|| is characterized by the dimension-free two-sided estimate 𝔼{||Σ̂-Σ||/||Σ||}≍√(r(Σ)/n)+r(Σ)/n, using the so-called effective rank r(Σ)≡tr(Σ)/||Σ||. The goal of this paper is to provide a dimension-free first-order characterization for ||Σ̂-Σ||. We show that |𝔼{||Σ̂-Σ||/||Σ||}/𝔼sup_α∈ [0,1]{(α+n^-1/2𝒢_Σ(h;α))^2-α^2}- 1| ≤C/√(r(Σ)), where {𝒢_Σ(h;α): α∈ [0,1]} are (stochastic) Gaussian widths over spherical slices of the (standardized) Σ-ellipsoid, playing the role of a first-order analogue to the zeroth-order characteristic r(Σ). As an immediate application of the first-order characterization, we obtain a version of the Koltchinskii-Lounici bound with optimal constants. In the more special context of spiked covariance models, our first-order characterization reveals a new phase transition of ||Σ̂-Σ|| that exhibits qualitatively different behavior compared to the BBP phase transitional behavior of ||Σ̂||. A similar phase transition is also proved for the associated eigenvector.
READ FULL TEXT