Understanding the dynamics of message passing algorithms: a free probability heuristics

02/03/2020

∙

We use freeness assumptions of random matrix theory to analyze the dynamical behavior of inference algorithms for probabilistic models with dense coupling matrices in the limit of large systems. For a toy Ising model, we are able to recover previous results such as the property of vanishing effective memories and the analytical convergence rate of the algorithm.

READ FULL TEXT

Understanding the dynamics of message passing algorithms: a free probability heuristics

Exact solution to the random sequential dynamics of a message passing algorithm

Analysis of Random Sequential Message Passing Algorithms for Approximate Inference

A Dynamical Mean-Field Theory for Learning in Restricted Boltzmann Machines

High-temperature Expansions and Message Passing Algorithms

Convergent Dynamics for Solving the TAP Equations of Ising Models with Arbitrary Rotation Invariant Coupling Matrices

Analysis of Bayesian Inference Algorithms by the Dynamical Functional Approach

Modeling and Estimation of Discrete-Time Reciprocal Processes via Probabilistic Graphical Models

Understanding the dynamics of message passing algorithms: a free probability heuristics

Related Research

Exact solution to the random sequential dynamics of a message passing algorithm

Analysis of Random Sequential Message Passing Algorithms for Approximate Inference

A Dynamical Mean-Field Theory for Learning in Restricted Boltzmann Machines

High-temperature Expansions and Message Passing Algorithms

Convergent Dynamics for Solving the TAP Equations of Ising Models with Arbitrary Rotation Invariant Coupling Matrices

Analysis of Bayesian Inference Algorithms by the Dynamical Functional Approach

Modeling and Estimation of Discrete-Time Reciprocal Processes via Probabilistic Graphical Models