An online algorithm for contrastive Principal Component Analysis

11/14/2022
by   Siavash Golkar, et al.
0

Finding informative low-dimensional representations that can be computed efficiently in large datasets is an important problem in data analysis. Recently, contrastive Principal Component Analysis (cPCA) was proposed as a more informative generalization of PCA that takes advantage of contrastive learning. However, the performance of cPCA is sensitive to hyper-parameter choice and there is currently no online algorithm for implementing cPCA. Here, we introduce a modified cPCA method, which we denote cPCA*, that is more interpretable and less sensitive to the choice of hyper-parameter. We derive an online algorithm for cPCA* and show that it maps onto a neural network with local learning rules, so it can potentially be implemented in energy efficient neuromorphic hardware. We evaluate the performance of our online algorithm on real datasets and highlight the differences and similarities with the original formulation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/14/2020

Probabilistic Contrastive Principal Component Analysis

Dimension reduction is useful for exploratory data analysis. In many app...
research
01/29/2022

Deep Contrastive Learning is Provably (almost) Principal Component Analysis

We show that Contrastive Learning (CL) under a family of loss functions ...
research
09/20/2017

Contrastive Principal Component Analysis

We present a new technique called contrastive principal component analys...
research
06/27/2021

Interpretable Network Representation Learning with Principal Component Analysis

We consider the problem of interpretable network representation learning...
research
07/09/2020

Contrastive Multiple Correspondence Analysis (cMCA): Applying the Contrastive Learning Method to Identify Political Subgroups

Ideal point estimation and dimensionality reduction have long been utili...
research
02/01/2022

Surrogate Gradients Design

Surrogate gradient (SG) training provides the possibility to quickly tra...
research
02/11/2021

Online Deterministic Annealing for Classification and Clustering

We introduce an online prototype-based learning algorithm for clustering...

Please sign up or login with your details

Forgot password? Click here to reset