. The GAMP allows general measurement channels (including non-linear channels) to be used. Due to its Bayes optimality as well as low computational complexity, and more importantly, asymptotical accuracy of state evolution (SE), GAMP has attracted more and more attention in domains like compressive sensing, image processing, Bayesian learning, statistical physics, low-rank matrix estimation, mmWave channel estimation, spatial modulation, user activity and signal detection in random access, orthogonal frequency division multiplexing analog-to-digital converters system, sparse superposition codes, etc[11, 10, 9, 8, 7, 6, 5, 4].
is an alternative message passing rule that deals with general non-Gaussian probability distribution functions (PDFs). EP projects thea-posteriori14, 15, 16, 17, 18]. The potential connection between AMP and EP was first shown in [19, 20], in which the fixed points of EP and AMP were shown to be consistent. An EP-based AMP was proposed in . Recently, Ma and Ping proposed an orthogonal AMP for general unitarily-invariant measurement matrices, and showed that the optimal MMSE OAMP is equivalent to MMSE EP [22, 24, 23]. These works hint at the conceptual equivalence between EP and AMP. In , Meng et al. first gave a rigorous derivation of AMP based on a dense graph-based EP by making some approximations in large system limit. Based on the results in 
, the authors further provided a unified Bayesian inference framework for the extension of AMP and VAMP to the generalized linear model[26, 27]. Another form of EP-based derivation for MMSE GAMP was illustrated in . More recently, the connection between EP and the max-sum GAMP was built in .
In [2, 1, 3], the authors used Taylor expansion and second-order approximation for the non-linear constraints of the general measurement channel. In this paper, we adopt a different approach, in which the general non-linear constraints are solved by an easily understandable EP rule, which has the same form as the GMP rule (for the linear constraints). The only difference between EP and GMP is that the a-posteriori calculation is replaced by a non-linear MMSE estimation, which makes EP more efficient in solving the non-linear problems than GMP. As a result, the whole general measurement problem is solved by the unified “GMP-like” rule. By neglecting the high-order infinitesimal terms, the EP-MPA is proven to be equivalent to GAMP. Furthermore, for additive white Gaussian noise (AWGN) measurement channels, the EP-MPA is proven to be equivalent to AMP. These results offer a new insight into GAMP and AMP, and may provide hints to build new MPAs for more general non-linear networks.
We credit [25, 26, 27, 28, 29] for the work on the consistency between EP and GAMP/AMP. However, this correspondence firstly provides a unified “GMP-like” rule for the MPAs in solving general measurement channels.
Ii Problem Formulation
GAMP considers a system given in Fig. 1, where , , and are subjected to a linear function , and and are subjected to symbol-wise transfer probability function and respectively. In addition, has i.i.d. Gaussian components . The goal of GAMP is to iteratively recover and given and , which is equivalent to estimate the marginal probability below
where is a Dirac delta function. However, exact calculation of (1) has intractable complexity for large scale problems.
For more general with finite , we can rewrite the system to , where and . Then, all the results in this paper are still valid by replacing with . For example, if , we replace by to make the results of this paper be valid.
Iii EP-Based Message Passing Algorithm
Fig. 2 gives a Forney-style factor graph of the system in (1), where edges denote variables, and nodes denote the related constraints: , , and . MPA  is a method to iteratively compute the marginal probability. Since the high-dimensional integration is distributively calculated by local message passing, it has a low complexity. Next, we briefly introduce EP [12, 13].
Iii-a Expectation Propagation
Let the a-priori message be with , and a constraint of . EP updates
where and .
Relation to Standard GMP: In fact, when the constraint is a linear and Gaussian111For example, is a Gaussian constraint of , and (given , and the distribution of ) is a linear constraint of ., EP in (2) is the exact GMP. For example, if is a Gaussian constraint , the a posteriori probability is Gaussian and given by
which can be rewritten to
That is, the information contained in the a-posteriori message is equal to the sum information contained in the a-priori message and the extrinsic message. This principle has been widely used in modern channel coding and sum-product algorithm. For example, the extrinsic message can be calculated by removing the a-priori message from the a-posteriori message.
If is non-Gaussian, EP in (2) is not equal to GMP, i.e., (2) and (6) are not equivalent, i.e., “information equation” in (7) does not hold any more. In general, EP could provide more useful information than EMP (or Turbo) for non-Gaussian , i.e., the following “information inequality” holds:
Intuition of EP: In general, the a posteriori probability (APP) estimation is the optimal local estimation since it fully exploits the a-priori (or input) message, but it will cause correlation problem in the iterative process. To avoid the correlation problem in the iteration, Turbo principle discards the a-priori message in the estimation, but this results in performance loss since the a-priori message is not exploited. EP makes a good tradeoff between the APP and Turbo, i.e., the a-priori message is partly used to improve the estimation and the correlation problem is also avoided. Due to these reasons, EP could have a better performance than EMP.
Fig. 3 shows the message passing illustration for the problem, where be the messages (mean and variance for ) passing from VN to XCN, for from XCN to VN, for from VN to SN, for from SN to VN, and for from SN to ZCN, and for from ZCN to SN. Next, we derive the message passing algorithm based on the expectation propagation principle under a unified “GMP-like” rule.
Iii-B A Unified “GMP-like” EP-MPA
Fig. 3 illustrates the EP-MPA, where ZCN and XCN denote the constraint nodes of and respectively. The message updates at variable node (VN) and sum node (SN) are GMP, while ZCN and XCN are EP.
Step I (SN ZCN): Since and , from central limit theorem (CLT), we have , where
with initialization , .
Step II (ZCN SN): Message update at ZCN for uses EP with constraints and :
Step III (SN VN): The constraints at -th SN are , and . Message update at SN for VN are:
where and .
Step IV (VN XCN): The constraints at -th VN are . Message update at VN are:
Step V (XCN VN): Message update at XCN to VN for uses EP with constraints and , i.e., for each ,
Step VI (VN SN): The constraints at -th VN are and . Message update at VN for SN are:
where and .
We abandon the auxiliary variables , and have
Therefore, we obtain a unified “GMP-like” EP-MPA, and the above steps are summarized in Algorithm 1.
Iv Equivalence between EP and GAMP/AMP
The equivalence of EP and AMP is firstly derived in , based on which [26, 27] further proposed a unified Bayesian inference framework for the extension of AMP and VAMP to the generalized linear model. Another form of EP-based derivation for MMSE GAMP was illustrated in . In , the max-sum GAMP was built by EP. In this section, we derive the MMSE GAMP and MMSE AMP with some approximations on the unified “GMP-like” EP-MPA in Algorithm 1.
Iv-a Connection with GAMP
For simplicity, we define
For , we have
First, we have since the a-priori message does not increase the conditional variance. In addition, from the symmetry of the system, . Therefore, we have .
Message update in (12) can be rewritten as
See APPENDIX A.
Message update (12) can be rewritten as
See APPENDIX B.
EP-MPA is equivalent to MMSE GAMP.
For balance systems, we have and . Therefore, the MMSE GAMP can be further simplified to
where and .
Iv-B Connection with AMP
In AMP, from , we have
EP-MPA can be rewritten to AMP.
See APPENDIX C.
V Numerical Results
We study a clipped compressed sensing problem where follows a symbol-wise Bernoulli-Gaussian distribution, i.e. ,