DeepAI AI Chat
Log In Sign Up

Estimation of variance components, heritability and the ridge penalty in high-dimensional generalized linear models

by   Jurre R. Veerman, et al.

For high-dimensional linear regression models, we review and compare several estimators of variances τ^2 and σ^2 of the random slopes and errors, respectively. These variances relate directly to ridge regression penalty λ and heritability index h^2, often used in genetics. Direct and indirect estimators of these, either based on cross-validation (CV) or maximum marginal likelihood (MML), are also discussed. The comparisons include several cases of covariate matrix X_n × p, with p ≫ n, such as multi-collinear covariates and data-derived ones. In addition, we study robustness against departures from the model such as sparse instead of dense effects and non-Gaussian errors. An example on weight gain data with genomic covariates confirms the good performance of MML compared to CV. Several extensions are presented. First, to the high-dimensional linear mixed effects model, with REML as an alternative to MML. Second, to the conjugate Bayesian setting, which proves to be a good alternative. Third, and most prominently, to generalized linear models for which we derive a computationally efficient MML estimator by re-writing the marginal likelihood as an n-dimensional integral. For Poisson and Binomial ridge regression, we demonstrate the superior accuracy of the resulting MML estimator of λ as compared to CV. Software is provided to enable reproduction of all results presented here.


page 1

page 2

page 3

page 4


Penalization-induced shrinking without rotation in high dimensional GLM regression: a cavity analysis

In high dimensional regression, where the number of covariates is of the...

Fast cross-validation for multi-penalty ridge regression

Prediction based on multiple high-dimensional data types needs to accoun...

Revisiting complexity and the bias-variance tradeoff

The recent success of high-dimensional models, such as deep neural netwo...

Tuning-free ridge estimators for high-dimensional generalized linear models

Ridge estimators regularize the squared Euclidean lengths of parameters....

A Nonparametric Bayesian Method for Clustering of High-Dimensional Mixed Dataset

Motivation: Advances in next-generation sequencing (NGS) methods have en...

High-dimensional regression with potential prior information on variable importance

There are a variety of settings where vague prior information may be ava...

High heritability does not imply accurate prediction under the small additive effects hypothesis

Genome-Wide Association Studies (GWAS) explain only a small fraction of ...