EigenGame: PCA as a Nash Equilibrium

10/01/2020
by   Ian Gemp, et al.
6

We present a novel view on principal component analysis (PCA) as a competitive game in which each approximate eigenvector is controlled by a player whose goal is to maximize their own utility function. We analyze the properties of this PCA game and the behavior of its gradient based updates. The resulting algorithm which combines elements from Oja's rule with a generalized Gram-Schmidt orthogonalization is naturally decentralized and hence parallelizable through message passing. We demonstrate the scalability of the algorithm with experiments on large image datasets and neural network activations. We discuss how this new view of PCA as a differentiable game can lead to further algorithmic developments and insights.

READ FULL TEXT

page 9

page 16

research
05/23/2020

Principal Component Analysis Based on Tℓ_1-norm Maximization

Classical principal component analysis (PCA) may suffer from the sensiti...
research
06/16/2017

Self-adaptive node-based PCA encodings

In this paper we propose an algorithm, Simple Hebbian PCA, and prove tha...
research
07/09/2019

All Sparse PCA Models Are Wrong, But Some Are Useful. Part I: Computation of Scores, Residuals and Explained Variance

Sparse Principal Component Analysis (sPCA) is a popular matrix factoriza...
research
06/04/2021

PCA Initialization for Approximate Message Passing in Rotationally Invariant Models

We study the problem of estimating a rank-1 signal in the presence of ro...
research
11/26/2011

Learning a Factor Model via Regularized PCA

We consider the problem of learning a linear factor model. We propose a ...
research
03/01/2015

Phase Transitions in Sparse PCA

We study optimal estimation for sparse principal component analysis when...
research
06/09/2023

Extending Kernel PCA through Dualization: Sparsity, Robustness and Fast Algorithms

The goal of this paper is to revisit Kernel Principal Component Analysis...

Please sign up or login with your details

Forgot password? Click here to reset