Approximate Message Passing for orthogonally invariant ensembles: Multivariate non-linearities and spectral initialization

by   Xinyi Zhong, et al.

We study a class of Approximate Message Passing (AMP) algorithms for symmetric and rectangular spiked random matrix models with orthogonally invariant noise. The AMP iterates have fixed dimension K ≥ 1, a multivariate non-linearity is applied in each AMP iteration, and the algorithm is spectrally initialized with K super-critical sample eigenvectors. We derive the forms of the Onsager debiasing coefficients and corresponding AMP state evolution, which depend on the free cumulants of the noise spectral distribution. This extends previous results for such models with K=1 and an independent initialization. Applying this approach to Bayesian principal components analysis, we introduce a Bayes-OAMP algorithm that uses as its non-linearity the posterior mean conditional on all preceding AMP iterates. We describe a practical implementation of this algorithm, where all debiasing and state evolution parameters are estimated from the observed data, and we illustrate the accuracy and stability of this approach in simulations.


page 1

page 2

page 3

page 4


Approximate Message Passing algorithms for rotationally invariant matrices

Approximate Message Passing (AMP) algorithms have seen widespread use ac...

Universality of Approximate Message Passing Algorithms

We consider a broad class of Approximate Message Passing (AMP) algorithm...

PCA Initialization for Approximate Message Passing in Rotationally Invariant Models

We study the problem of estimating a rank-1 signal in the presence of ro...

Macroscopic Analysis of Vector Approximate Message Passing in a Model Mismatch Setting

Vector approximate message passing (VAMP) is an efficient approximate in...

Approximate Message Passing with a Colored Aliasing Model for Variable Density Fourier Sampled Images

The Approximate Message Passing (AMP) algorithm efficiently reconstructs...

A Non-Asymptotic Framework for Approximate Message Passing in Spiked Models

Approximate message passing (AMP) emerges as an effective iterative para...

Local convexity of the TAP free energy and AMP convergence for Z2-synchronization

We study mean-field variational Bayesian inference using the TAP approac...