DeepAI AI Chat
Log In Sign Up

A note on the variance in principal component regression

by   Bert van der Veen, et al.

Principal component regression is a popular method to use when the predictor matrix in a regression is of reduced column rank. It has been proposed to stabilize computation under such conditions, and to improve prediction accuracy by reducing variance of the least squares estimator for the regression slopes. However, it presents the added difficulty of having to determine which principal components to include in the regression. I provide arguments against selecting the principal components by the magnitude of their associated eigenvalues, by examining the estimator for the residual variance, and by examining the contribution of the residual variance to the variance of the estimator for the regression slopes. I show that when a principal component is omitted from the regression that is important in explaining the response variable, the residual variance is overestimated, so that the variance of the estimator for the regression slopes can be higher than that of the ordinary least squares estimator.


page 1

page 2

page 3

page 4


How many variables should be entered in a principal component regression equation?

We study least squares linear regression over N uncorrelated Gaussian fe...

Sparse principal component regression with adaptive loading

Principal component regression (PCR) is a two-stage procedure that selec...

Supervised Principal Component Regression for Functional Response with High Dimensional Predictors

We propose a supervised principal component regression method for relati...

Principal Component Projection and Regression in Nearly Linear Time through Asymmetric SVRG

Given a data matrix A∈R^n × d, principal component projection (PCP) and ...

Faster Principal Component Regression and Stable Matrix Chebyshev Approximation

We solve principal component regression (PCR), up to a multiplicative ac...

Errors-in-variables Fréchet Regression with Low-rank Covariate Approximation

Fréchet regression has emerged as a promising approach for regression an...

Who Votes for Library Bonds? A Principal Component Exploration

Previous research has shown a relationship between voter characteristics...