DeepAI AI Chat
Log In Sign Up

Inference with Deep Generative Priors in High Dimensions

by   Parthe Pandit, et al.

Deep generative priors offer powerful models for complex-structured data, such as images, audio, and text. Using these priors in inverse problems typically requires estimating the input and/or hidden signals in a multi-layer deep neural network from observation of its output. While these approaches have been successful in practice, rigorous performance analysis is complicated by the non-convex nature of the underlying optimization problems. This paper presents a novel algorithm, Multi-Layer Vector Approximate Message Passing (ML-VAMP), for inference in multi-layer stochastic neural networks. ML-VAMP can be configured to compute maximum a priori (MAP) or approximate minimum mean-squared error (MMSE) estimates for these networks. We show that the performance of ML-VAMP can be exactly predicted in a certain high-dimensional random limit. Furthermore, under certain conditions, ML-VAMP yields estimates that achieve the minimum (i.e., Bayes-optimal) MSE as predicted by the replica method. In this way, ML-VAMP provides a computationally efficient method for multi-layer inference with an exact performance characterization and testable conditions for optimality in the large-system limit.


Asymptotics of MAP Inference in Deep Networks

Deep generative priors are a powerful tool for reconstruction problems w...

Inference in Multi-Layer Networks with Matrix-Valued Unknowns

We consider the problem of inferring the input and hidden variables of a...

Inference in Deep Networks in High Dimensions

Deep generative networks provide a powerful tool for modeling complex da...

Multi-layer State Evolution Under Random Convolutional Design

Signal recovery under generative neural network priors has emerged as a ...

Estimation for High-Dimensional Multi-Layer Generalized Linear Model – Part I: The Exact MMSE Estimator

This two-part work considers the minimum means square error (MMSE) estim...

Estimation for High-Dimensional Multi-Layer Generalized Linear Model – Part II: The ML-GAMP Estimator

This is Part II of a two-part work on the estimation for a multi-layer g...

Approximate Message Passing for Multi-Layer Estimation in Rotationally Invariant Models

We consider the problem of reconstructing the signal and the hidden vari...