Optimal selection of a common subset of covariates for different regressions

by   David Azriel, et al.

Given a regression dataset of size n, most of the classical model selection literature treats the problem of selecting a subset of covariates to be used for prediction of future responses. In this paper we assume that a sample of J regression datasets of different sizes from a population of datasets having the same set of covariates and responses is observed. The goal is to select, for each n, a single subset of covariates to be used for prediction of future responses for any dataset of size n from the population (which may or may not be in the sample of datasets). The regression coefficients used in the prediction are estimated using the n observations consisting of covariates and responses in the sample for which prediction of future responses is to be done, and thus they differ across different samples. For example, if the response is a diagnosis, and the covariates are medical background variables and measurements, the goal is to select a standard set of measurements for different clinics, say, where each clinic may estimate and use its own coefficients for prediction (depending on local conditions, prevalence, etc.). The selected subset naturally depends on the sample size n, with a larger sample size allowing a more elaborate model. Since we consider prediction for any (or a randomly chosen) dataset in the population, it is natural to consider random covariates. If the population consists of datasets that are similar, our approach amounts to borrowing information, leading to a subset selection that is efficient for prediction. On the other hand, if the datasets are dissimilar, then our goal is to find a "compromise" subset of covariates for the different regressions.


page 1

page 2

page 3

page 4


Model Selection Consistency for Cointegrating Regressions

We study the asymptotic properties of the adaptive Lasso in cointegratio...

Model Selection for High-Dimensional Regression under the Generalized Irrepresentability Condition

In the high-dimensional regression model a response variable is linearly...

Competing Models

Different agents compete to predict a variable of interest related to a ...

Conformal Prediction for Network-Assisted Regression

An important problem in network analysis is predicting a node attribute ...

"Predicting" after peeking into the future: Correcting a fundamental flaw in the SAOM – TERGM comparison of Leifeld and Cranmer (2019)

We review the empirical comparison of SAOMs and TERGMs by Leifeld and Cr...

Fitting Probabilistic Index Models on Large Datasets

Recently, Thas et al. (2012) introduced a new statistical model for the ...

Subset selection for linear mixed models

Linear mixed models (LMMs) are instrumental for regression analysis with...

Please sign up or login with your details

Forgot password? Click here to reset