Fast Markov chain Monte Carlo for high dimensional Bayesian regression models with shrinkage priors

03/16/2019
by   Rui Jin, et al.
0

In the past decade, many Bayesian shrinkage models have been developed for linear regression problems where the number of covariates, p, is large. Computing the intractable posterior are often done with three-block Gibbs samplers (3BG), based on representing the shrinkage priors as scale mixtures of Normal distributions. An alternative computing tool is a state of the art Hamiltonian Monte Carlo (HMC) method, which can be easily implemented in the Stan software. However, we found both existing methods to be inefficient and often impractical for large p problems. Following the general idea of Rajaratnam et al. (2018), we propose two-block Gibbs samplers (2BG) for three commonly used shrinkage models, namely, the Bayesian group lasso, the Bayesian sparse group lasso and the Bayesian fused lasso models. We demonstrate with simulated and real data examples that the Markov chains underlying 2BG's converge much faster than that of 3BG's, and no worse than that of HMC. At the same time, the computing costs of 2BG's per iteration are as low as that of 3BG's, and can be several orders of magnitude lower than that of HMC. As a result, the newly proposed 2BG is the only practical computing solution to do Bayesian shrinkage analysis for datasets with large p. Further, we provide theoretical justifications for the superior performance of 2BG's. First, we establish geometric ergodicity (GE) of Markov chains associated with the 2BG for each of the three Bayesian shrinkage models, and derive quantitative upper bounds for their geometric convergence rates. Secondly, we show that the Markov operators corresponding to the 2BG of the Bayesian group lasso and the Bayesian sparse group lasso are trace class, respectively, whereas that of the corresponding 3BG are not even Hilbert-Schmidt.

READ FULL TEXT
research
04/16/2018

Trace class Markov chains for the Normal-Gamma Bayesian shrinkage model

High-dimensional data, where the number of variables exceeds or is compa...
research
12/09/2020

Coupled Markov chain Monte Carlo for high-dimensional regression with Half-t priors

Continuous shrinkage priors are commonly used in Bayesian analysis of hi...
research
01/02/2021

Geometric ergodicity of Gibbs samplers for the Horseshoe and its regularized variants

The Horseshoe is a widely used and popular continuous shrinkage prior fo...
research
11/18/2017

Fast Monte Carlo Markov chains for Bayesian shrinkage models with random effects

When performing Bayesian data analysis using a general linear mixed mode...
research
05/16/2019

The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions

Discovering interaction effects on a response of interest is a fundament...
research
05/29/2019

Bayesian Dynamic Fused LASSO

The new class of Markov processes is proposed to realize the flexible sh...
research
09/17/2022

Geometric ergodicity of Gibbs samplers for Bayesian error-in-variable regression

We consider Bayesian error-in-variable (EIV) linear regression accountin...

Please sign up or login with your details

Forgot password? Click here to reset