Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing

by   Ramji Venkataramanan, et al.

We consider the problem of signal estimation in generalized linear models defined via rotationally invariant design matrices. Since these matrices can have an arbitrary spectral distribution, this model is well suited to capture complex correlation structures which often arise in applications. We propose a novel family of approximate message passing (AMP) algorithms for signal estimation, and rigorously characterize their performance in the high-dimensional limit via a state evolution recursion. Assuming knowledge of the design matrix spectrum, our rotationally invariant AMP has complexity of the same order as the existing AMP for Gaussian matrices; it also recovers the existing AMP as a special case. Numerical results showcase a performance close to Vector AMP (which is conjectured to be Bayes-optimal in some settings), but obtained with a much lower complexity, as the proposed algorithm does not require a computationally expensive singular value decomposition.


page 1

page 2

page 3

page 4


Generalized Memory Approximate Message Passing

Generalized approximate message passing (GAMP) is a promising technique ...

Generalized Approximate Survey Propagation for High-Dimensional Estimation

In Generalized Linear Estimation (GLE) problems, we seek to estimate a s...

Bayesian Nonlinear Function Estimation with Approximate Message Passing

In many areas, massive amounts of data are collected and analyzed in ord...

Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models

We study the problem of recovering an unknown signal x given measurement...

Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (or : How to Prove Kabashima's Replica Formula)

There has been a recent surge of interest in the study of asymptotic rec...

Macroscopic Analysis of Vector Approximate Message Passing in a Model Mismatch Setting

Vector approximate message passing (VAMP) is an efficient approximate in...

Additivity of Information in Multilayer Networks via Additive Gaussian Noise Transforms

Multilayer (or deep) networks are powerful probabilistic models based on...