Derivation of Coupled PCA and SVD Learning Rules from a Newton Zero-Finding Framework

03/25/2020
by   Ralf Möller, et al.
6

In coupled learning rules for PCA (principal component analysis) and SVD (singular value decomposition), the update of the estimates of eigenvectors or singular vectors is influenced by the estimates of eigenvalues or singular values, respectively. This coupled update mitigates the speed-stability problem since the update equations converge from all directions with approximately the same speed. A method to derive coupled learning rules from information criteria by Newton optimization is known. However, these information criteria have to be designed, offer no explanatory value, and can only impose Euclidean constraints on the vector estimates. Here we describe an alternative approach where coupled PCA and SVD learning rules can systematically be derived from a Newton zero-finding framework. The derivation starts from an objective function, combines the equations for its extrema with arbitrary constraints on the vector estimates, and solves the resulting vector zero-point equation using Newton's zero-finding method. To demonstrate the framework, we derive PCA and SVD learning rules with constant Euclidean length or constant sum of the vector estimates.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/28/2022

Derivation of Learning Rules for Coupled Principal Component Analysis in a Lagrange-Newton Framework

We describe a Lagrange-Newton framework for the derivation of learning r...
research
06/28/2019

FameSVD: Fast and Memory-efficient Singular Value Decomposition

We propose a novel algorithm to perform the Singular Value Decomposition...
research
10/21/2022

Robust Singular Values based on L1-norm PCA

Singular-Value Decomposition (SVD) is a ubiquitous data analysis method ...
research
05/24/2020

Derivation of Symmetric PCA Learning Rules from a Novel Objective Function

Neural learning rules for principal component / subspace analysis (PCA /...
research
05/22/2019

Sparse Equisigned PCA: Algorithms and Performance Bounds in the Noisy Rank-1 Setting

Singular value decomposition (SVD) based principal component analysis (P...
research
07/27/2023

PCA, SVD, and Centering of Data

The research detailed in this paper scrutinizes Principal Component Anal...
research
02/10/2017

PCA in Data-Dependent Noise (Correlated-PCA): Nearly Optimal Finite Sample Guarantees

We study Principal Component Analysis (PCA) in a setting where a part of...

Please sign up or login with your details

Forgot password? Click here to reset