On Column Selection in Approximate Kernel Canonical Correlation Analysis

02/05/2016
by   Weiran Wang, et al.
0

We study the problem of column selection in large-scale kernel canonical correlation analysis (KCCA) using the Nyström approximation, where one approximates two positive semi-definite kernel matrices using "landmark" points from the training set. When building low-rank kernel approximations in KCCA, previous work mostly samples the landmarks uniformly at random from the training set. We propose novel strategies for sampling the landmarks non-uniformly based on a version of statistical leverage scores recently developed for kernel ridge regression. We study the approximation accuracy of the proposed non-uniform sampling strategy, develop an incremental algorithm that explores the path of approximation ranks and facilitates efficient model selection, and derive the kernel stability of out-of-sample mapping for our method. Experimental results on both synthetic and real-world datasets demonstrate the promise of our method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/19/2020

Kernel Ridge Regression Using Importance Sampling with Application to Seismic Response Prediction

Scalable kernel methods, including kernel ridge regression, often rely o...
research
05/24/2016

Recursive Sampling for the Nyström Method

We give the first algorithm for kernel Nyström approximation that runs i...
research
02/20/2020

Diversity sampling is an implicit regularization for kernel methods

Kernel methods have achieved very good performance on large scale regres...
research
05/21/2018

Relating Leverage Scores and Density using Regularized Christoffel Functions

Statistical leverage scores emerged as a fundamental tool for matrix ske...
research
10/31/2018

On Fast Leverage Score Sampling and Optimal Learning

Leverage score sampling provides an appealing way to perform approximate...
research
10/11/2019

A General Scoring Rule for Randomized Kernel Approximation with Application to Canonical Correlation Analysis

Random features has been widely used for kernel approximation in large-s...
research
05/29/2019

Nyström landmark sampling and regularized Christoffel functions

Selecting diverse and important items from a large set is a problem of i...

Please sign up or login with your details

Forgot password? Click here to reset