DeepAI AI Chat
Log In Sign Up

Diffusion approximations of Oja's online principal component analysis

by   Jian-Guo Liu, et al.

Oja's algorithm of principal component analysis (PCA) has been one of the methods utilized in practice to reduce dimension. In this paper, we focus on the convergence property of the discrete algorithm. To realize that, we view the algorithm as a stochastic process on the parameter space and semi-group. We approximate it by SDEs, and prove large time convergence of the SDEs to ensure its performance. This process is completed in three steps. First, the discrete algorithm can be viewed as a semigroup: S^kΟ†=𝔼[Ο†(𝐖(k))]. Second, we construct stochastic differential equations (SDEs) on the Stiefel manifold, i.e. the diffusion approximation, to approximate the semigroup. By proving the weak convergence, we verify that the algorithm is 'close to' the SDEs. Finally, we use the reversibility of the SDEs to prove long-time convergence.


page 1

page 2

page 3

page 4

βˆ™ 08/29/2018

Diffusion Approximations for Online Principal Component Estimation and Global Convergence

In this paper, we propose to adopt the diffusion approximation tools to ...
βˆ™ 03/16/2016

Near-Optimal Stochastic Approximation for Online Principal Component Estimation

Principal component analysis (PCA) has been a prominent tool for high-di...
βˆ™ 03/12/2020

On the Convergence of the Dynamic Inner PCA Algorithm

Dynamic inner principal component analysis (DiPCA) is a powerful method ...
βˆ™ 03/31/2021

On the Optimality of the Oja's Algorithm for Online PCA

In this paper we analyze the behavior of the Oja's algorithm for online/...
βˆ™ 03/31/2011

Auto-associative models, nonlinear Principal component analysis, manifolds and projection pursuit

In this paper, auto-associative models are proposed as candidates to the...
βˆ™ 07/18/2020

Improved Convergence Speed of Fully Symmetric Learning Rules for Principal Component Analysis

Fully symmetric learning rules for principal component analysis can be d...
βˆ™ 03/07/2018

Sketching for Principal Component Regression

Principal component regression (PCR) is a useful method for regularizing...