DeepAI AI Chat
Log In Sign Up

Principal Component Projection Without Principal Component Analysis

by   Roy Frostig, et al.
Stanford University

We show how to efficiently project a vector onto the top principal components of a matrix, without explicitly computing these components. Specifically, we introduce an iterative algorithm that provably computes the projection using few calls to any black-box routine for ridge regression. By avoiding explicit principal component analysis (PCA), our algorithm is the first with no runtime dependence on the number of top principal components. We show that it can be used to give a fast iterative method for the popular principal component regression problem, giving the first major runtime improvement over the naive method of combining PCA with regression. To achieve our results, we first observe that ridge regression can be used to obtain a "smooth projection" onto the top principal components. We then sharpen this approximation to true projection using a low-degree polynomial approximation to the matrix step function. Step function approximation is a topic of long-term interest in scientific computing. We extend prior theory by constructing polynomials with simple iterative structure and rigorously analyzing their behavior under limited precision.


page 1

page 2

page 3

page 4


Faster Principal Component Regression and Stable Matrix Chebyshev Approximation

We solve principal component regression (PCR), up to a multiplicative ac...

Principal Component Projection and Regression in Nearly Linear Time through Asymmetric SVRG

Given a data matrix A∈R^n × d, principal component projection (PCP) and ...

A Tutorial on Principal Component Analysis

Principal component analysis (PCA) is a mainstay of modern data analysis...

A Study on a User-Controlled Radial Tour for Variable Importance in High-Dimensional Data

Principal component analysis is a long-standing go-to method for explori...

Principal Model Analysis Based on Partial Least Squares

Motivated by the Bagging Partial Least Squares (PLS) and Principal Compo...

Modeling of Individual HRTFs based on Spatial Principal Component Analysis

Head-related transfer function (HRTF) plays an important role in the con...

Advanced Variations of Two-Dimensional Principal Component Analysis for Face Recognition

The two-dimensional principal component analysis (2DPCA) has become one ...