Sharper dimension-free bounds on the Frobenius distance between sample covariance and its expectation
We study properties of a sample covariance estimate Ξ£= (π_1 π_1^β€ + β¦ + π_n π_n^β€) / n, where π_1, β¦, π_n are i.i.d. random elements in β^d with πΌπ_1 = 0, πΌπ_1 π_1^β€ = Ξ£. We derive dimension-free bounds on the squared Frobenius norm of (Ξ£- Ξ£) under reasonable assumptions. For instance, we show that | Ξ£- Ξ£_ F^2 - πΌΞ£- Ξ£_ F^2| = πͺ(Tr(Ξ£^2) / n) with overwhelming probability, which is a significant improvement over the existing results. This leads to a bound the ratio Ξ£- Ξ£_ F^2 / πΌΞ£- Ξ£_ F^2 with a sharp leading constant when the effective rank π(Ξ£) = Tr(Ξ£) / Ξ£ and n / π(Ξ£)^6 tend to infinity: Ξ£- Ξ£_ F^2 / πΌΞ£- Ξ£_ F^2 = 1 + πͺ(1 / π(Ξ£)).
READ FULL TEXT