A Simple Derivation of AMP and its State Evolution via First-Order Cancellation

07/09/2019
by   Phil Schniter, et al.
0

We consider the linear regression problem, where the goal is to recover the vector x∈R^n from measurements y=Ax+w∈R^m under known matrix A and unknown noise w. For large i.i.d. sub-Gaussian A, the approximate message passing (AMP) algorithm is precisely analyzable through a state-evolution (SE) formalism, which furthermore shows that AMP is Bayes optimal in certain regimes. The rigorous SE proof, however, is long and complicated. And, although the AMP algorithm can be derived as an approximation of loop belief propagation (LBP), this viewpoint provides little insight into why large i.i.d. A matrices are important for AMP. In this work, we provide a heuristic derivation of AMP and its state evolution, based on the idea of "first-order cancellation," that provides insights missing from the LBP derivation while being much shorter than the rigorous SE proof.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/26/2016

Learning and Free Energies for Vector Approximate Message Passing

Vector approximate message passing (VAMP) is a computationally simple ap...
research
01/19/2022

A Concise Tutorial on Approximate Message Passing

High-dimensional signal recovery of standard linear regression is a key ...
research
03/25/2020

Rigorous State Evolution Analysis for Approximate Message Passing with Side Information

A common goal in many research areas is to reconstruct an unknown signal...
research
09/24/2021

Graph-based Approximate Message Passing Iterations

Approximate-message passing (AMP) algorithms have become an important el...
research
05/23/2018

A Simple Re-Derivation of Onsager's Solution of the 2D Ising Model using Experimental Mathematics

In this case study, we illustrate the great potential of experimental ma...
research
02/01/2019

An Analysis of State Evolution for Approximate Message Passing with Side Information

A common goal in many research areas is to reconstruct an unknown signal...
research
08/12/2022

The planted XY model: thermodynamics and inference

In this paper we study a fully connected planted spin glass named the pl...

Please sign up or login with your details

Forgot password? Click here to reset