Scalable Gaussian-process regression and variable selection using Vecchia approximations

02/25/2022
by   Jian Cao, et al.
0

Gaussian process (GP) regression is a flexible, nonparametric approach to regression that naturally quantifies uncertainty. In many applications, the number of responses and covariates are both large, and a goal is to select covariates that are related to the response. For this setting, we propose a novel, scalable algorithm, coined VGPR, which optimizes a penalized GP log-likelihood based on the Vecchia GP approximation, an ordered conditional approximation from spatial statistics that implies a sparse Cholesky factor of the precision matrix. We traverse the regularization path from strong to weak penalization, sequentially adding candidate covariates based on the gradient of the log-likelihood and deselecting irrelevant covariates via a new quadratic constrained coordinate descent algorithm. We propose Vecchia-based mini-batch subsampling, which provides unbiased gradient estimators. The resulting procedure is scalable to millions of responses and thousands of covariates. Theoretical analysis and numerical studies demonstrate the improved scalability and accuracy relative to existing methods.

READ FULL TEXT

page 13

page 16

research
12/20/2017

Laplace approximation and the natural gradient for Gaussian process regression with the heteroscedastic Student-t model

This paper considers the Laplace method to derive approximate inference ...
research
03/04/2022

High-dimensional Censored Regression via the Penalized Tobit Likelihood

The Tobit model has long been the standard method for regression with a ...
research
04/08/2021

Approximate Bayesian inference from noisy likelihoods with Gaussian process emulated MCMC

We present an efficient approach for doing approximate Bayesian inferenc...
research
05/03/2019

Parallel Gaussian process surrogate method to accelerate likelihood-free inference

We consider Bayesian inference when only a limited number of noisy log-l...
research
02/20/2019

Gaussian Process Priors for Dynamic Paired Comparison Modelling

Dynamic paired comparison models, such as Elo and Glicko, are frequently...
research
10/03/2021

Hierarchical Gaussian Process Models for Regression Discontinuity/Kink under Sharp and Fuzzy Designs

We propose nonparametric Bayesian estimators for causal inference exploi...
research
01/04/2023

A Scalable Gaussian Process for Large-Scale Periodic Data

The periodic Gaussian process (PGP) has been increasingly used to model ...

Please sign up or login with your details

Forgot password? Click here to reset