Estimation for High-Dimensional Multi-Layer Generalized Linear Model – Part II: The ML-GAMP Estimator

by   Qiuyun Zou, et al.

This is Part II of a two-part work on the estimation for a multi-layer generalized linear model (ML-GLM) in large system limits. In Part I, we had analyzed the asymptotic performance of an exact MMSE estimator, and obtained a set of coupled equations that could characterize its MSE performance. To work around the implementation difficulty of the exact estimator, this paper continues to propose an approximate solution, ML-GAMP, which could be derived by blending a moment-matching projection into the Gaussian approximated loopy belief propagation. The ML-GAMP estimator is then shown to enjoy a great simplicity in its implementation, where its per-iteration complexity is as low as GAMP. Further analysis on its asymptotic performance also reveals that, in large system limits, its dynamical MSE behavior is fully characterized by a set of simple one-dimensional iterating equations, termed state evolution (SE). Interestingly, this SE of ML-GAMP share exactly the same fixed points with an exact MMSE estimator whose fixed points were obtained in Part I via a replica analysis. Given the Bayes-optimality of the exact implementation, this proposed estimator (if converged) is optimal in the MSE sense.



There are no comments yet.


page 1

page 2

page 3

page 4


Estimation for High-Dimensional Multi-Layer Generalized Linear Model – Part I: The Exact MMSE Estimator

This two-part work considers the minimum means square error (MMSE) estim...

Multi-Layer Bilinear Generalized Approximate Message Passing

In this paper, we extend the bilinear generalized approximate message pa...

Replica Analysis for Generalized Linear Regression with IID Row Prior

Different from a typical independent identically distributed (IID) eleme...

Inference with Deep Generative Priors in High Dimensions

Deep generative priors offer powerful models for complex-structured data...

Limiting free energy of multi-layer generalized linear models

We compute the high-dimensional limit of the free energy associated with...

Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models

We study the problem of recovering an unknown signal x given measurement...

Estimation of Mittag-Leffler Parameters

We propose a procedure for estimating the parameters of the Mittag-Leffl...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.