Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models

03/05/2019
by   Ray Bai, et al.
0

We introduce the spike-and-slab group lasso (SSGL) for Bayesian estimation and variable selection in linear regression with grouped variables. We further extend the SSGL to sparse generalized additive models (GAMs), thereby introducing the first nonparametric variant of the spike-and-slab lasso methodology. Our model simultaneously performs group selection and estimation, while our fully Bayes treatment of the mixture proportion allows for model complexity control and automatic self-adaptivity to different levels of sparsity. We develop theory to uniquely characterize the global posterior mode under the SSGL and introduce highly efficient coordinate ascent algorithms for maximum a posteriori (MAP) estimation. We further employ de-biasing methods to provide uncertainty quantification of our MAP estimates. Thus, implementation of our model avoids the computational intensiveness of Markov chain Monte Carlo (MCMC) in high dimensions. We derive posterior concentration rates for both grouped linear regression and sparse GAMs when the number of covariates grows at nearly exponential rate with sample size. Finally, we illustrate our methodology through extensive simulations and data analysis.

READ FULL TEXT
research
07/15/2019

Fast Algorithms and Theory for High-Dimensional Bayesian Varying Coefficient Models

Nonparametric varying coefficient (NVC) models are widely used for model...
research
05/29/2015

On the Computational Complexity of High-Dimensional Bayesian Variable Selection

We study the computational complexity of Markov chain Monte Carlo (MCMC)...
research
01/30/2023

Convergence of uncertainty estimates in Ensemble and Bayesian sparse model discovery

Sparse model identification enables nonlinear dynamical system discovery...
research
07/14/2020

A Unified Computational and Theoretical Framework for High-Dimensional Bayesian Additive Models

We introduce a general framework for estimation and variable selection i...
research
09/19/2023

Group Spike and Slab Variational Bayes

In this manuscript we introduce Group Spike-and-slab Variational Bayes (...
research
12/03/2020

On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regression

This paper presents the use of spike-and-slab (SS) priors for discoverin...
research
10/27/2021

Spike-and-Slab Generalized Additive Models and Scalable Algorithms for High-Dimensional Data

There are proposals that extend the classical generalized additive model...

Please sign up or login with your details

Forgot password? Click here to reset