Tensor Canonical Correlation Analysis

06/12/2019
by   You-Lin Chen, et al.
0

In many applications, such as classification of images or videos, it is of interest to develop a framework for tensor data instead of ad-hoc way of transforming data to vectors due to the computational and under-sampling issues. In this paper, we study canonical correlation analysis by extending the framework of two dimensional analysis (Lee and Choi, 2007) to tensor-valued data. Instead of adopting the iterative algorithm provided in Lee and Choi (2007), we propose an efficient algorithm, called the higher-order power method, which is commonly used in tensor decomposition and more efficient for large-scale setting. Moreover, we carefully examine theoretical properties of our algorithm and establish a local convergence property via the theory of Lojasiewicz's inequalities. Our results fill a missing, but crucial, part in the literature on tensor data. For practical applications, we further develop (a) an inexact updating scheme which allows us to use the state-of-the-art stochastic gradient descent algorithm, (b) an effective initialization scheme which alleviates the problem of local optimum in non-convex optimization, and (c) an extension for extracting several canonical components. Empirical analyses on challenging data including gene expression, air pollution indexes in Taiwan, and electricity demand in Australia, show the effectiveness and efficiency of the proposed methodology.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/06/2015

Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition

We analyze stochastic gradient descent for optimizing non-convex functio...
research
02/10/2023

Tensor Generalized Canonical Correlation Analysis

Regularized Generalized Canonical Correlation Analysis (RGCCA) is a gene...
research
02/09/2015

Tensor Canonical Correlation Analysis for Multi-view Dimension Reduction

Canonical correlation analysis (CCA) has proven an effective tool for tw...
research
11/15/2016

The Power of Normalization: Faster Evasion of Saddle Points

A commonly used heuristic in non-convex optimization is Normalized Gradi...
research
03/18/2020

NeCPD: An Online Tensor Decomposition with Optimal Stochastic Gradient Descent

Multi-way data analysis has become an essential tool for capturing under...
research
11/20/2018

Gen-Oja: A Simple and Efficient Algorithm for Streaming Generalized Eigenvector Computation

In this paper, we study the problems of principal Generalized Eigenvecto...
research
02/26/2018

Shampoo: Preconditioned Stochastic Tensor Optimization

Preconditioned gradient methods are among the most general and powerful ...

Please sign up or login with your details

Forgot password? Click here to reset