DeepAI
Log In Sign Up

Near-Optimal Stochastic Approximation for Online Principal Component Estimation

03/16/2016
by   Chris Junchi Li, et al.
0

Principal component analysis (PCA) has been a prominent tool for high-dimensional data analysis. Online algorithms that estimate the principal component by processing streaming data are of tremendous practical and theoretical interests. Despite its rich applications, theoretical convergence analysis remains largely open. In this paper, we cast online PCA into a stochastic nonconvex optimization problem, and we analyze the online PCA algorithm as a stochastic approximation iteration. The stochastic approximation iteration processes data points incrementally and maintains a running estimate of the principal component. We prove for the first time a nearly optimal finite-sample error bound for the online PCA algorithm. Under the subgaussian assumption, we show that the finite-sample error bound closely matches the minimax information lower bound.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/29/2018

Diffusion Approximations for Online Principal Component Estimation and Global Convergence

In this paper, we propose to adopt the diffusion approximation tools to ...
12/28/2020

Stochastic Approximation for Online Tensorial Independent Component Analysis

Independent component analysis (ICA) has been a popular dimension reduct...
10/08/2018

Find the dimension that counts: Fast dimension estimation and Krylov PCA

High dimensional data and systems with many degrees of freedom are often...
05/15/2020

Non-Sparse PCA in High Dimensions via Cone Projected Power Iteration

In this paper, we propose a cone projected power iteration algorithm to ...
06/26/2019

Principal Component Analysis for Multivariate Extremes

The first order behavior of multivariate heavy-tailed random vectors abo...
12/19/2018

Fast Botnet Detection From Streaming Logs Using Online Lanczos Method

Botnet, a group of coordinated bots, is becoming the main platform of ma...
02/10/2017

PCA in Data-Dependent Noise (Correlated-PCA): Nearly Optimal Finite Sample Guarantees

We study Principal Component Analysis (PCA) in a setting where a part of...