Scalable Bayesian regression in high dimensions with multiple data sources

by   Konstantinos Perrakis, et al.

Current applications of high-dimensional regression in biomedicine often involve multiple sources or types of covariates. We propose methodology for this setting, focusing on the "wide data" regime with large total dimensionality p and sample size n<<p. As a starting point, we formulate a flexible ridge-type prior with shrinkage levels that are specific to each data type or source. These multiple shrinkage levels are set automatically using empirical Bayes. Importantly, all estimation, including setting of shrinkage levels, can be formulated mainly in terms of inner product matrices of size n x n, rendering computation fast and scalable in the wide data setting, even for millions of predictors, and the resulting procedures are free of user-set tuning parameters. We consider sparse extensions via constrained minimization of a certain Kullback-Leibler divergence. This includes a relaxed variant that scales to large p, allows adaptive and source-specific shrinkage and has a closed-form solution. We compare these approaches to standard high-dimensional methods in a simulation study based on biological data.


Flexible co-data learning for high-dimensional prediction

Clinical research often focuses on complex traits in which many variable...

High-dimensional general linear hypothesis tests via non-linear spectral shrinkage

We are interested in testing general linear hypotheses in a high-dimensi...

Scalable Sparse Cox's Regression for Large-Scale Survival Data via Broken Adaptive Ridge

This paper develops a new sparse Cox regression method for high-dimensio...

Asymptotics for high-dimensional covariance matrices and quadratic forms with applications to the trace functional and shrinkage

We establish large sample approximations for an arbitray number of bilin...

Adaptive penalization in high-dimensional regression and classification with external covariates using variational Bayes

Penalization schemes like Lasso or ridge regression are routinely used t...

Forecasting in Big Data Environments: an Adaptable and Automated Shrinkage Estimation of Neural Networks (AAShNet)

This paper considers improved forecasting in possibly nonlinear dynamic ...

Please sign up or login with your details

Forgot password? Click here to reset