DeepAI AI Chat
Log In Sign Up

Fast Ridge Regression with Randomized Principal Component Analysis and Gradient Descent

by   Yichao Lu, et al.
University of Pennsylvania

We propose a new two stage algorithm LING for large scale regression problems. LING has the same risk as the well known Ridge Regression under the fixed design setting and can be computed much faster. Our experiments have shown that LING performs well in terms of both prediction accuracy and computational efficiency compared with other large scale regression algorithms like Gradient Descent, Stochastic Gradient Descent and Principal Component Regression on both simulated and real datasets.


page 1

page 2

page 3

page 4


A note on the prediction error of principal component regression

We analyse the prediction error of principal component regression (PCR) ...

A Risk Comparison of Ordinary Least Squares vs Ridge Regression

We compare the risk of ridge regression to a simple variant of ordinary ...

Distributed Principal Component Analysis with Limited Communication

We study efficient distributed algorithms for the fundamental problem of...

Faster Principal Component Regression and Stable Matrix Chebyshev Approximation

We solve principal component regression (PCR), up to a multiplicative ac...

A General Framework for Analyzing Stochastic Dynamics in Learning Algorithms

We present a general framework for analyzing high-probability bounds for...

Understanding Machine-learned Density Functionals

Kernel ridge regression is used to approximate the kinetic energy of non...