Covariate Selection Based on a Model-free Approach to Linear Regression with Exact Probabilities

02/03/2022
by   Laurie Davies, et al.
0

In this paper we give a completely new approach to the problem of covariate selection in linear regression. A covariate or a set of covariates is included only if it is better in the sense of least squares than the same number of Gaussian covariates consisting of i.i.d. N(0,1) random variables. The Gaussian P-value is defined as the probability that the Gaussian covariates are better. It is given in terms of the Beta distribution, it is exact and it holds for all data making it model-free free. The covariate selection procedures require only a cut-off value α for the Gaussian P-value: the default value in this paper is α=0.01. The resulting procedures are very simple, very fast, do not overfit and require only least squares. In particular there is no regularization parameter, no data splitting, no use of simulations, no shrinkage and no post selection inference is required. The paper includes the results of simulations, applications to real data sets and theorems on the asymptotic behaviour under the standard linear model. Here the step-wise procedure performs overwhelmingly better than any other procedure we are aware of. An R-package gausscov is available.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/05/2019

A Model-free Approach to Linear Least Squares Regression with Exact Probabilities and Applications to Covariate Selection

The classical model for linear regression is Y= xβ +σε with i.i.d. stan...
research
05/04/2018

Lasso, knockoff and Gaussian covariates: a comparison

Given data y and k covariates x_j one problem in linear regression is to...
research
12/16/2021

Linear Regression, Covariate Selection and the Failure of Modelling

It is argued that all model based approaches to the selection of covaria...
research
03/07/2019

Relaxing the Assumptions of Knockoffs by Conditioning

The recent paper Candès et al. (2018) introduced model-X knockoffs, a me...
research
07/25/2018

A model-free approach to linear least squares regression with exact probabilities

In a regression setting with observation vector y ∈ R^n and given finite...
research
11/07/2018

Asymptotic conditional inference via a Steining of selection probabilities

Many scientific studies are modeled as hierarchical procedures where the...
research
02/08/2021

A test for comparing conditional ROC curves with multidimensional covariates

The comparison of Receiver Operating Characteristic (ROC) curves is freq...

Please sign up or login with your details

Forgot password? Click here to reset