Estimator of Prediction Error Based on Approximate Message Passing for Penalized Linear Regression

02/20/2018
by   Ayaka Sakata, et al.
0

We propose an estimator of prediction error using an approximate message passing (AMP) algorithm that can be applied to a broad range of sparse penalties. Following Stein's lemma, the estimator of the generalized degrees of freedom, which is a key quantity for the construction of the estimator of the prediction error, is calculated at the AMP fixed point. The resulting form of the AMP-based estimator does not depend on the penalty function, and its value can be further improved by considering the correlation between predictors. The proposed estimator is asymptotically unbiased when the components of the predictors and response variables are independently generated according to a Gaussian distribution. We examine the behaviour of the estimator for real data under nonconvex sparse penalties, where Akaike's information criterion does not correspond to an unbiased estimator of the prediction error. The model selected by the proposed estimator is close to that which minimizes the true prediction error.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/02/2021

Asymptotic Statistical Analysis of Sparse Group LASSO via Approximate Message Passing Algorithm

Sparse Group LASSO (SGL) is a regularized model for high-dimensional lin...
research
02/02/2023

Role of Bootstrap Averaging in Generalized Approximate Message Passing

Generalized approximate message passing (GAMP) is a computationally effi...
research
06/26/2022

Prediction Errors for Penalized Regressions based on Generalized Approximate Message Passing

We discuss the prediction accuracy of assumed statistical models in term...
research
09/15/2015

Sparse Multinomial Logistic Regression via Approximate Message Passing

For the problem of multi-class linear classification and feature selecti...
research
07/31/2015

Robustness in sparse linear models: relative efficiency based on robust approximate message passing

Understanding efficiency in high dimensional linear models is a longstan...
research
05/31/2021

Neural message passing for joint paratope-epitope prediction

Antibodies are proteins in the immune system which bind to antigens to d...
research
11/08/2017

Approximate message passing for nonconvex sparse regularization with stability and asymptotic analysis

We analyze linear regression problem with a nonconvex regularization cal...

Please sign up or login with your details

Forgot password? Click here to reset