AdaOja: Adaptive Learning Rates for Streaming PCA

05/28/2019
by   Amelia Henriksen, et al.
0

Oja's algorithm has been the cornerstone of streaming methods in Principal Component Analysis (PCA) since it was first proposed in 1982. However, Oja's algorithm does not have a standardized choice of learning rate (step size) that both performs well in practice and truly conforms to the online streaming setting. In this paper, we propose a new learning rate scheme for Oja's method called AdaOja. This new algorithm requires only a single pass over the data and does not depend on knowing properties of the data set a priori. AdaOja is a novel variation of the Adagrad algorithm to Oja's algorithm in the single eigenvector case and extended to the multiple eigenvector case. We demonstrate for dense synthetic data, sparse real-world data and dense real-world data that AdaOja outperforms common learning rate choices for Oja's method. We also show that AdaOja performs comparably to state-of-the-art algorithms (History PCA and Streaming Power Method) in the same streaming PCA setting.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/15/2018

History PCA: A New Algorithm for Streaming PCA

In this paper we propose a new algorithm for streaming principal compone...
research
06/04/2015

Rivalry of Two Families of Algorithms for Memory-Restricted Streaming PCA

We study the problem of recovering the subspace spanned by the first k p...
research
07/17/2018

An Acceleration Scheme for Memory Limited, Streaming PCA

In this paper, we propose an acceleration scheme for online memory-limit...
research
04/03/2018

Average performance analysis of the stochastic gradient method for online PCA

This paper studies the complexity of the stochastic gradient algorithm f...
research
01/04/2020

Distributed Stochastic Algorithms for High-rate Streaming Principal Component Analysis

This paper considers the problem of estimating the principal eigenvector...
research
03/10/2015

An Adaptive Online HDP-HMM for Segmentation and Classification of Sequential Data

In the recent years, the desire and need to understand sequential data h...
research
11/04/2019

ODE-Inspired Analysis for the Biological Version of Oja's Rule in Solving Streaming PCA

Oja's rule [Oja, Journal of mathematical biology 1982] is a well-known b...

Please sign up or login with your details

Forgot password? Click here to reset