Finding Linear Structure in Large Datasets with Scalable Canonical Correlation Analysis

06/26/2015
by   Zhuang Ma, et al.
0

Canonical Correlation Analysis (CCA) is a widely used spectral technique for finding correlation structures in multi-view datasets. In this paper, we tackle the problem of large scale CCA, where classical algorithms, usually requiring computing the product of two huge matrices and huge matrix decomposition, are computationally and storage expensive. We recast CCA from a novel perspective and propose a scalable and memory efficient Augmented Approximate Gradient (AppGrad) scheme for finding top k dimensional canonical subspace which only involves large matrix multiplying a thin matrix of width k and small matrix decomposition of dimension k× k. Further, AppGrad achieves optimal storage complexity O(k(p_1+p_2)), compared with classical algorithms which usually require O(p_1^2+p_2^2) space to store two dense whitening matrices. The proposed scheme naturally generalizes to stochastic optimization regime, especially efficient for huge datasets where batch algorithms are prohibitive. The online property of stochastic AppGrad is also well suited to the streaming scenario, where data comes sequentially. To the best of our knowledge, it is the first stochastic algorithm for CCA. Experiments on four real data sets are provided to show the effectiveness of the proposed methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/16/2014

Large scale canonical correlation analysis with iterative least squares

Canonical Correlation Analysis (CCA) is a widely used statistical tool w...
research
02/22/2017

Stochastic Approximation for Canonical Correlation Analysis

We study canonical correlation analysis (CCA) as a stochastic optimizati...
research
02/09/2015

Tensor Canonical Correlation Analysis for Multi-view Dimension Reduction

Canonical correlation analysis (CCA) has proven an effective tool for tw...
research
01/09/2020

D-GCCA: Decomposition-based Generalized Canonical Correlation Analysis for Multiple High-dimensional Datasets

Modern biomedical studies often collect multiple types of high-dimension...
research
01/16/2019

Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization

This work considers the problem of computing the canonical polyadic deco...
research
03/25/2020

Generalized Canonical Correlation Analysis: A Subspace Intersection Approach

Generalized Canonical Correlation Analysis (GCCA) is an important tool t...
research
12/06/2013

Dual coordinate solvers for large-scale structural SVMs

This manuscript describes a method for training linear SVMs (including b...

Please sign up or login with your details

Forgot password? Click here to reset