Bayesian Bootstrap Spike-and-Slab LASSO

by   Lizhen Nie, et al.

The impracticality of posterior sampling has prevented the widespread adoption of spike-and-slab priors in high-dimensional applications. To alleviate the computational burden, optimization strategies have been proposed that quickly find local posterior modes. Trading off uncertainty quantification for computational speed, these strategies have enabled spike-and-slab deployments at scales that would be previously unfeasible. We build on one recent development in this strand of work: the Spike-and-Slab LASSO procedure of Ročková and George (2018). Instead of optimization, however, we explore multiple avenues for posterior sampling, some traditional and some new. Intrigued by the speed of Spike-and-Slab LASSO mode detection, we explore the possibility of sampling from an approximate posterior by performing MAP optimization on many independently perturbed datasets. To this end, we explore Bayesian bootstrap ideas and introduce a new class of jittered Spike-and-Slab LASSO priors with random shrinkage targets. These priors are a key constituent of the Bayesian Bootstrap Spike-and-Slab LASSO (BB-SSL) method proposed here. BB-SSL turns fast optimization into approximate posterior sampling. Beyond its scalability, we show that BB-SSL has a strong theoretical support. Indeed, we find that the induced pseudo-posteriors contract around the truth at a near-optimal rate in sparse normal-means and in high-dimensional regression. We compare our algorithm to the traditional Stochastic Search Variable Selection (under Laplace priors) as well as many state-of-the-art methods for shrinkage priors. We show, both in simulations and on real data, that our method fares superbly in these comparisons, often providing substantial computational gains.


Bayesian Joint Spike-and-Slab Graphical Lasso

In this article, we propose a new class of priors for Bayesian inference...

Bayesian Lesion Estimation with a Structured Spike-and-Slab Prior

Neural demyelination and brain damage accumulated in white matter appear...

Fast Exact Bayesian Inference for Sparse Signals in the Normal Sequence Model

We consider exact algorithms for Bayesian inference with model selection...

Double spike Dirichlet priors for structured weighting

Assigning weights to a large pool of objects is a fundamental task in a ...

Bayesian l_0 Regularized Least Squares

Bayesian l_0-regularized least squares provides a variable selection tec...

Scalable Spike-and-Slab

Spike-and-slab priors are commonly used for Bayesian variable selection,...

l_1-ball Prior: Uncertainty Quantification with Exact Zeros

Lasso and l_1-regularization play a dominating role in high dimensional ...

Please sign up or login with your details

Forgot password? Click here to reset