Nearly optimal Bayesian Shrinkage for High Dimensional Regression

12/24/2017
by   Qifan Song, et al.
0

During the past decade, shrinkage priors have received much attention in Bayesian analysis of high-dimensional data. In this paper, we study the problem for high-dimensional linear regression models. We show that if the shrinkage prior has a heavy and flat tail, and allocates a sufficiently large probability mass in a very small neighborhood of zero, then its posterior properties are as good as those of the spike-and-slab prior. While enjoying its efficiency in Bayesian computation, the shrinkage prior can lead to a nearly optimal contraction rate and selection consistency as the spike-and-slab prior. Our numerical results show that under posterior consistency, Bayesian methods can yield much better results in variable selection than the regularization methods, such as Lasso and SCAD. We also establish a Bernstein von-Mises type results comparable to Castillo et al (2015), this result leads to a convenient way to quantify uncertainties of the regression coefficient estimates, which has been beyond the ability of regularization methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/09/2019

Ultra High-dimensional Multivariate Posterior Contraction Rate Under Shrinkage Priors

In recent years, shrinkage priors have received much attention in high-d...
research
10/24/2020

Nearly Optimal Variational Inference for High Dimensional Regression with Shrinkage Priors

We propose a variational Bayesian (VB) procedure for high-dimensional li...
research
04/24/2019

Horseshoe Regularization for Machine Learning in Complex and Deep Models

Since the advent of the horseshoe priors for regularization, global-loca...
research
10/13/2020

Spike-and-Slab Meets LASSO: A Review of the Spike-and-Slab LASSO

High-dimensional data sets have become ubiquitous in the past few decade...
research
03/23/2022

Separating the Wheat from the Chaff: Bayesian Regularization in Dynamic Social Networks

In recent years there has been an increasing interest in the use of rela...
research
02/09/2023

Consistent Group selection using Global-local prior in High dimensional setup

We consider the problem of model selection when grouping structure is in...
research
04/11/2020

Bayesian Shrinkage towards Sharp Minimaxity

Shrinkage prior are becoming more and more popular in Bayesian modeling ...

Please sign up or login with your details

Forgot password? Click here to reset