Replicated Vector Approximate Message Passing For Resampling Problem

05/23/2019
by   Takashi Takahashi, et al.
0

Resampling techniques are widely used in statistical inference and ensemble learning, in which estimators' statistical properties are essential. However, existing methods are computationally demanding, because repetitions of estimation/learning via numerical optimization/integral for each resampled data are required. In this study, we introduce a computationally efficient method to resolve such problem: replicated vector approximate message passing. This is based on a combination of the replica method of statistical physics and an accurate approximate inference algorithm, namely the vector approximate message passing of information theory. The method provides tractable densities without repeating estimation/learning, and the densities approximately offer an arbitrary degree of the estimators' moment in practical time. In the experiment, we apply the proposed method to the stability selection method, which is commonly used in variable selection problems. The numerical results show its fast convergence and high approximation accuracy for problems involving both synthetic and real-world datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/19/2020

Semi-analytic approximate stability selection for correlated data in generalized linear models

We consider the variable selection problem of generalized linear models ...
research
05/05/2021

A unifying tutorial on Approximate Message Passing

Over the last decade or so, Approximate Message Passing (AMP) algorithms...
research
09/15/2015

Sparse Multinomial Logistic Regression via Approximate Message Passing

For the problem of multi-class linear classification and feature selecti...
research
04/12/2020

Active Sampling for Pairwise Comparisons via Approximate Message Passing and Information Gain Maximization

Pairwise comparison data arise in many domains with subjective assessmen...
research
01/09/2020

Macroscopic Analysis of Vector Approximate Message Passing in a Model Mismatch Setting

Vector approximate message passing (VAMP) is an efficient approximate in...
research
06/26/2022

Prediction Errors for Penalized Regressions based on Generalized Approximate Message Passing

We discuss the prediction accuracy of assumed statistical models in term...
research
05/13/2018

Accelerating Message Passing for MAP with Benders Decomposition

We introduce a novel mechanism to tighten the local polytope relaxation ...

Please sign up or login with your details

Forgot password? Click here to reset