Multi-layer State Evolution Under Random Convolutional Design

05/26/2022
by   Max Daniels, et al.
0

Signal recovery under generative neural network priors has emerged as a promising direction in statistical inference and computational imaging. Theoretical analysis of reconstruction algorithms under generative priors is, however, challenging. For generative priors with fully connected layers and Gaussian i.i.d. weights, this was achieved by the multi-layer approximate message (ML-AMP) algorithm via a rigorous state evolution. However, practical generative priors are typically convolutional, allowing for computational benefits and inductive biases, and so the Gaussian i.i.d. weight assumption is very limiting. In this paper, we overcome this limitation and establish the state evolution of ML-AMP for random convolutional layers. We prove in particular that random convolutional layers belong to the same universality class as Gaussian matrices. Our proof technique is of an independent interest as it establishes a mapping between convolutional matrices and spatially coupled sensing matrices used in coding theory.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/03/2022

Approximate Message Passing for Multi-Layer Estimation in Rotationally Invariant Models

We consider the problem of reconstructing the signal and the hidden vari...
research
11/08/2019

Inference with Deep Generative Priors in High Dimensions

Deep generative priors offer powerful models for complex-structured data...
research
04/24/2022

Signal Recovery with Non-Expansive Generative Network Priors

We study compressive sensing with a deep generative network prior. Initi...
research
09/21/2021

Stabilizing Elastic Weight Consolidation method in practical ML tasks and using weight importances for neural network pruning

This paper is devoted to the features of the practical application of El...
research
03/01/2019

Asymptotics of MAP Inference in Deep Networks

Deep generative priors are a powerful tool for reconstruction problems w...
research
06/20/2017

Inference in Deep Networks in High Dimensions

Deep generative networks provide a powerful tool for modeling complex da...
research
10/12/2022

Orthogonal Approximate Message-Passing for Spatially Coupled Systems

Orthogonal approximate message-passing (OAMP) is proposed for signal rec...

Please sign up or login with your details

Forgot password? Click here to reset