The TAP free energy for high-dimensional linear regression

03/14/2022
by   Jiaze Qiu, et al.
0

We derive a variational representation for the log-normalizing constant of the posterior distribution in Bayesian linear regression with a uniform spherical prior and an i.i.d. Gaussian design. We work under the "proportional" asymptotic regime, where the number of observations and the number of features grow at a proportional rate. This rigorously establishes the Thouless-Anderson-Palmer (TAP) approximation arising from spin glass theory, and proves a conjecture of Krzakala et. al. (2014) in the special case of the spherical prior.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/25/2021

Variational Inference in high-dimensional linear regression

We study high-dimensional Bayesian linear regression with product priors...
research
06/06/2023

Asymptotics of Bayesian Uncertainty Estimation in Random Features Regression

In this paper we compare and contrast the behavior of the posterior pred...
research
12/08/2021

Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression

We study a localized notion of uniform convergence known as an "optimist...
research
07/14/2021

Performance of Bayesian linear regression in a model with mismatch

For a model of high-dimensional linear regression with random design, we...
research
09/30/2020

Regress Consistently when Oblivious Outliers Overwhelm

We give a novel analysis of the Huber loss estimator for consistent robu...
research
03/08/2022

Asymptotic normality in linear regression with approximately sparse structure

In this paper we study the asymptotic normality in high-dimensional line...
research
03/24/2023

The limited-memory recursive variational Gaussian approximation (L-RVGA)

We consider the problem of computing a Gaussian approximation to the pos...

Please sign up or login with your details

Forgot password? Click here to reset