Gain with no Pain: Efficient Kernel-PCA by Nyström Sampling

07/11/2019
by   Nicholas Sterge, et al.
10

In this paper, we propose and study a Nyström based approach to efficient large scale kernel principal component analysis (PCA). The latter is a natural nonlinear extension of classical PCA based on considering a nonlinear feature map or the corresponding kernel. Like other kernel approaches, kernel PCA enjoys good mathematical and statistical properties but, numerically, it scales poorly with the sample size. Our analysis shows that Nyström sampling greatly improves computational efficiency without incurring any loss of statistical accuracy. While similar effects have been observed in supervised learning, this is the first such result for PCA. Our theoretical findings, which are also illustrated by numerical results, are based on a combination of analytic and concentration of measure techniques. Our study is more broadly motivated by the question of understanding the interplay between statistical and computational requirements for learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/15/2012

Kernel Principal Component Analysis and its Applications in Face Recognition and Active Shape Models

Principal component analysis (PCA) is a popular tool for linear dimensio...
research
05/19/2021

Statistical Optimality and Computational Efficiency of Nyström Kernel PCA

Kernel methods provide an elegant framework for developing nonlinear lea...
research
09/12/2021

Kernel PCA with the Nyström method

Kernel methods are powerful but computationally demanding techniques for...
research
02/16/2018

Inferring relevant features: from QFT to PCA

In many-body physics, renormalization techniques are used to extract asp...
research
08/27/2019

Statistical and Computational Trade-Offs in Kernel K-Means

We investigate the efficiency of k-means in terms of both statistical an...
research
11/26/2011

Learning a Factor Model via Regularized PCA

We consider the problem of learning a linear factor model. We propose a ...
research
12/21/2016

Robust Learning with Kernel Mean p-Power Error Loss

Correntropy is a second order statistical measure in kernel space, which...

Please sign up or login with your details

Forgot password? Click here to reset