Intuitive Joint Priors for Bayesian Linear Multilevel Models: The R2D2M2 prior

by   Javier Enrique Aguilar, et al.

The training of high-dimensional regression models on comparably sparse data is an important yet complicated topic, especially when there are many more model parameters than observations in the data. From a Bayesian perspective, inference in such cases can be achieved with the help of shrinkage prior distributions, at least for generalized linear models. However, real-world data usually possess multilevel structures, such as repeated measurements or natural groupings of individuals, which existing shrinkage priors are not built to deal with. We generalize and extend one of these priors, the R2D2 prior by Zhang et al. (2020), to linear multilevel models leading to what we call the R2D2M2 prior. The proposed prior enables both local and global shrinkage of the model parameters. It comes with interpretable hyperparameters, which we show to be intrinsically related to vital properties of the prior, such as rates of concentration around the origin, tail behavior, and amount of shrinkage the prior exerts. We offer guidelines on how to select the prior's hyperparameters by deriving shrinkage factors and measuring the effective number of non-zero model coefficients. Hence, the user can readily evaluate and interpret the amount of shrinkage implied by a specific choice of hyperparameters. Finally, we perform extensive experiments on simulated and real data, showing that our inference procedure for the prior is well calibrated, has desirable global and local regularization properties and enables the reliable and interpretable estimation of much more complex Bayesian multilevel models than was previously possible.


page 1

page 2

page 3

page 4


Adaptive Bayesian Shrinkage Estimation Using Log-Scale Shrinkage Priors

Global-local shrinkage hierarchies are an important, recent innovation i...

Sparse Bayesian Inference on Positive-valued Data using Global-local Shrinkage Priors

In various applications, we deal with high-dimensional positive-valued d...

Model selection for ecological community data using tree shrinkage priors

Researchers and managers model ecological communities to infer the bioti...

Horseshoe Regularization for Machine Learning in Complex and Deep Models

Since the advent of the horseshoe priors for regularization, global-loca...

Robust priors for regularized regression

Induction benefits from useful priors. Penalized regression approaches, ...

Dual-Space Analysis of the Sparse Linear Model

Sparse linear (or generalized linear) models combine a standard likeliho...

Subspace Shrinkage in Conjugate Bayesian Vector Autoregressions

Macroeconomists using large datasets often face the choice of working wi...

Please sign up or login with your details

Forgot password? Click here to reset