A note on the variance in principal component regression

01/04/2023
by   Bert van der Veen, et al.
0

Principal component regression is a popular method to use when the predictor matrix in a regression is of reduced column rank. It has been proposed to stabilize computation under such conditions, and to improve prediction accuracy by reducing variance of the least squares estimator for the regression slopes. However, it presents the added difficulty of having to determine which principal components to include in the regression. I provide arguments against selecting the principal components by the magnitude of their associated eigenvalues, by examining the estimator for the residual variance, and by examining the contribution of the residual variance to the variance of the estimator for the regression slopes. I show that when a principal component is omitted from the regression that is important in explaining the response variable, the residual variance is overestimated, so that the variance of the estimator for the regression slopes can be higher than that of the ordinary least squares estimator.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/04/2019

How many variables should be entered in a principal component regression equation?

We study least squares linear regression over N uncorrelated Gaussian fe...
research
02/26/2014

Sparse principal component regression with adaptive loading

Principal component regression (PCR) is a two-stage procedure that selec...
research
03/22/2021

Supervised Principal Component Regression for Functional Response with High Dimensional Predictors

We propose a supervised principal component regression method for relati...
research
10/15/2019

Principal Component Projection and Regression in Nearly Linear Time through Asymmetric SVRG

Given a data matrix A∈R^n × d, principal component projection (PCP) and ...
research
08/16/2016

Faster Principal Component Regression and Stable Matrix Chebyshev Approximation

We solve principal component regression (PCR), up to a multiplicative ac...
research
05/16/2023

Errors-in-variables Fréchet Regression with Low-rank Covariate Approximation

Fréchet regression has emerged as a promising approach for regression an...
research
06/26/2021

Who Votes for Library Bonds? A Principal Component Exploration

Previous research has shown a relationship between voter characteristics...

Please sign up or login with your details

Forgot password? Click here to reset