Replica Analysis for Generalized Linear Regression with IID Row Prior

by   Qiuyun Zou, et al.

Different from a typical independent identically distributed (IID) element assumption, this paper studies the estimation of IID row random matrix for the generalized linear model constructed by a linear mixing space and a row-wise mapping channel. The objective inference problem arises in many engineering fields, such as wireless communications, compressed sensing, and phase retrieval. We apply the replica method from statistical mechanics to analyze the exact minimum mean square error (MMSE) under the Bayes-optimal setting, in which the explicit replica symmetric solution of the exact MMSE estimator is obtained. Meanwhile, the input-output mutual information relation between the objective model and the equivalent single-vector system is established. To estimate the signal, we also propose a computationally efficient message passing based algorithm on expectation propagation (EP) perspective and analyze its dynamics. We verify that the asymptotic MSE of proposed algorithm predicted by its state evolution (SE) matches perfectly the exact MMSE estimator predicted by the replica method. That indicates, the optimal MSE error can be attained by the proposed algorithm if it has a unique fixed point.



There are no comments yet.


page 1

page 13


Multi-Layer Bilinear Generalized Approximate Message Passing

In this paper, we extend the bilinear generalized approximate message pa...

Bayes-Optimal Convolutional AMP

This paper proposes Bayes-optimal convolutional approximate message-pass...

A Concise Tutorial on Approximate Message Passing

High-dimensional signal recovery of standard linear regression is a key ...

Estimation for High-Dimensional Multi-Layer Generalized Linear Model – Part II: The ML-GAMP Estimator

This is Part II of a two-part work on the estimation for a multi-layer g...

Estimation for High-Dimensional Multi-Layer Generalized Linear Model – Part I: The Exact MMSE Estimator

This two-part work considers the minimum means square error (MMSE) estim...

The fundamental limits of sparse linear regression with sublinear sparsity

We establish exact asymptotic expressions for the normalized mutual info...

Sufficient Statistic Memory AMP

Approximate message passing (AMP) is a promising technique for unknown s...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.