Laplace Matching for fast Approximate Inference in Generalized Linear Models

05/07/2021
by   Marius Hobbhahn, et al.
0

Bayesian inference in generalized linear models (GLMs), i.e. Gaussian regression with non-Gaussian likelihoods, is generally non-analytic and requires computationally expensive approximations, such as sampling or variational inference. We propose an approximate inference framework primarily designed to be computationally cheap while still achieving high approximation quality. The concept, which we call Laplace Matching, involves closed-form, approximate, bi-directional transformations between the parameter spaces of exponential families. These are constructed from Laplace approximations under custom-designed basis transformations. The mappings can then be leveraged to effectively turn a latent Gaussian distribution into a conjugate prior for a rich class of observable variables. This effectively turns inference in GLMs into conjugate inference (with small approximation errors). We empirically evaluate the method in two different GLMs, showing approximation quality comparable to state-of-the-art approximate inference techniques at a drastic reduction in computational cost. More specifically, our method has a cost comparable to the very first step of the iterative optimization usually employed in standard GLM inference.

READ FULL TEXT

page 14

page 18

research
10/10/2022

Sampling-based inference for large linear models, with application to linearised Laplace

Large-scale linear models are ubiquitous throughout machine learning, wi...
research
01/15/2019

Mixed Variational Inference

The Laplace approximation has been one of the workhorses of Bayesian inf...
research
03/16/2020

Laplace approximation for fast Bayesian inference in generalized additive models based on penalized regression splines

Generalized additive models (GAMs) are a well-established statistical to...
research
07/19/2023

VITS : Variational Inference Thomson Sampling for contextual bandits

In this paper, we introduce and analyze a variant of the Thompson sampli...
research
08/01/2020

Convergence of Sparse Variational Inference in Gaussian Processes Regression

Gaussian processes are distributions over functions that are versatile a...
research
04/10/2022

Parallelized integrated nested Laplace approximations for fast Bayesian inference

There is a growing demand for performing larger-scale Bayesian inference...
research
12/14/2020

Approximate Laplace approximations for scalable model selection

We propose the approximate Laplace approximation (ALA) to evaluate integ...

Please sign up or login with your details

Forgot password? Click here to reset