Convolutional Approximate Message-Passing

I Introduction

Approximate message-passing (AMP) [1] is a low-complexity algorithm of signal recovery in compressed sensing [2, 3]. When the sensing matrix has independent and identically distributed (i.i.d.) zero-mean sub-Gaussian elements [4, 5], AMP was proved to be asymptotically Bayes-optimal in a certain region of the compression rate. However, AMP fails to converge when the sensing matrix is ill-conditioned [6] or has non-zero mean [7].

To solve this convergence issue of AMP, orthogonal AMP (OAMP) [8] and vector AMP (VAMP) [9] were proposed. OAMP and VAMP are equivalent to each other. The Bayes-optimal version of OAMP/VAMP was originally proposed by Opper and Winther [10]. OAMP/VAMP was proved to be asymptotically Bayes-optimal when the sensing matrix is orthogonally invariant [9, 11]

. However, OAMP/VAMP has high complexity unless the singular-value decomposition (SVD) of the sensing matrix can be computed efficiently.

This letter proposes a novel message-passing (MP) algorithm that solves both the convergence issue of AMP and the complexity issue of OAMP/VAMP. The proposed MP uses the same matched filter as AMP while OAMP/VAMP utilizes a linear minimum mean-square error (LMMSE) filter. Furthermore, it performs the so-called Onsager correction via a convolution of messages in all preceding iterations while AMP and OAMP/VAMP have correction terms that depend only on messages in the latest iteration. Thus, the proposed MP is called convolutional AMP (CAMP).

The tap coefficients in the convolution are determined so as to realize the asymptotic Gaussianity of the estimation errors of CAMP. For that purpose, they are defined such that a general error model proposed in

[12] contains the error model of CAMP asymptotically. Since the asymptotic Gaussianity in the general error model has been proved for any orthogonally invariant sensing matrix [12]

, the estimation errors of CAMP are asymptotically Gaussian-distributed. Numerical simulations for ill-conditioned sensing matrices show that CAMP can achieve performance comparable to OAMP/VAMP in spite of complexity comparable to AMP.

Ii Measurement Model

Consider the $M$ -dimensional linear measurements $y \in R^{M}$ of an unknown $N$ -dimensional sparse signal vector $x \in R^{N}$ ,

y = A x + w, w \sim N (0, σ^{2} I_{M}) .

(1)

In (1), $A \in R^{M \times N}$ denotes a known sensing matrix. The vector $w$ is additive white Gaussian noise (AWGN) with covariance $σ^{2} I_{M}$ . The triple $(A, x, w)$

is independent random variables. For simplicity, the signal vector

x

is assumed to have i.i.d. elements with zero mean and unit variance. Furthermore, the power normalization

N^{- 1} E [∥ A ∥^{2}] = 1

is assumed.

An important assumption is the right-orthogonal invariance of $A$ : In the SVD $A = U Σ V^{T}$ , the $N \times N$ orthogonal matrix $V$ is independent of $U Σ$ and Haar-distributed [13]. This class of matrices contains zero-mean i.i.d. Gaussian matrices.

As an additional technical assumption, the empirical eigenvalue distribution of

A^{T} A

converges almost surely to a deterministic distribution with a compact support in the large system limit, in which

M

and

N

tend to infinity while the compression rate

δ = M / N

is kept

O (1)

. Let

μ_{k}

denote the

k

th moment of the empirical eigenvalue distribution,

μ_{k} = \frac{1}{N} T r (Λ^{k}),

(2)

with $Λ = Σ^{T} Σ$ . The technical assumption implies that any moment $μ_{k}$ converges almost surely in the large system limit. In particular, the power normalization $N^{- 1} E [∥ A ∥^{2}] = 1$ implies $μ_{1} a . s . \to 1$ in the large system limit.

Iii Convolutional AMP

Iii-a Algorithm

The so-called Onsager correction is used to guarantee the asymptotic Gaussianity of the estimation errors before thresholding in each iteration of MP. The Onsager correction in AMP depends only on a message in the latest iteration. While AMP is a low-complexity algorithm, the Onsager correction in AMP fails to guarantee the asymptotic Gaussianity, with the only exception of zero-mean i.i.d. sensing matrices [4, 5].

The proposed CAMP has Onsager correction applicable to all right-orthogonally invariant sensing matrices. The correction term is a convolution of messages in all preceding iterations. Thus, the proposed MP is called convolutional AMP.

Let $x_{t} \in R^{N}$ denote an estimator of $x$ in iteration $t$ of CAMP. The estimator $x_{t}$ is recursively given by

x_{t + 1} = f_{t} (x_{t} + A^{T} z_{t}),

(3)

z_{t} = y - A x_{t} + t - 1 \sum τ = 0 ξ_{τ}^{(t - 1)} g_{t - τ - 1}^{(1)} z_{τ},

(4)

ξ_{t}^{(t^{'})} = t^{'} \prod τ = t ⟨ f_{τ}^{'} (x_{τ} + A^{T} z_{τ}) ⟩,

(5)

with $x_{0} = 0$ . In the CAMP, ${f_{t} : R \to R}$ are a sequence of Lipschitz-continuous thresholding functions. For any function $f : R \to R$ , $f (v)$ represents the element-wise application of $f$ to a vector $v$ , i.e. $[f (v)]_{n} = f ([v]_{n})$ . The notional convention $\sum_{τ = 0}^{- 1} \dots = 0$ is used in (4). The notation $⟨ v ⟩ = N^{- 1} \sum_{n = 1}^{N} v_{n}$ denotes the arithmetic mean of the elements of $v = (v_{1}, \dots, v_{N})^{T}$ . The CAMP reduces to conventional AMP in the case of $g_{0}^{(1)} = δ^{- 1}$ and $g_{t}^{(1)} = 0$ for all $t > 0$ .

To define the tap coefficients ${g_{t}^{(1)}}$ in the CAMP, consider a discrete-time dynamical system ${g_{t}^{(k)} : k, t = 0, 1, \dots}$ ,

g_{0}^{(k)} = μ_{k + 1} - μ_{k},

(6)

g_{1}^{(k)} = g_{0}^{(k)} - g_{0}^{(k + 1)} + g_{0}^{(1)} μ_{k + 1},

(7)

	$g_{t}^{(k)} =$	$g_{t - 1}^{(k)} - g_{t - 1}^{(k + 1)} + t - 1 \sum τ = 1 g_{t - τ - 1}^{(1)} (g_{τ}^{(k)} - g_{τ - 1}^{(k)})$			(8)
		$+ g_{t - 1}^{(1)} μ_{k + 1} for t \geq 2,$			(8)

where $μ_{k}$ denotes the $k$ th moment (2) of the empirical eigenvalue distribution of $A^{T} A$ .

In a practical implementation, the moment sequence should be replaced by the asymptotic one in the large system limit. This replacement implies that the complexity to compute ${g_{t}^{(1)}}$ can be independent of the system size if the asymptotic eigenvalue distribution of $A^{T} A$ has a closed-form expression.

AMP	OAMP/VAMP	CAMP
$O (t M N)$	$O (M^{2} N + t M N)$	$O (t M N + t^{2} M)$

TABLE I: Complexity in

M \leq N

and the number of iterations

t

The computational complexity of the CAMP, AMP, and OAMP/VAMP is compared in Table I. The complexity of AMP is dominated by matrix-vector multiplication. The first term for OAMP/VAMP is the worst-case complexity of the SVD of $A$ . The second term for the CAMP is due to computation of the Onsager correction term. As long as the number of iterations $t$ is much smaller than $M$ and $N$ , the complexity of the CAMP is comparable to that of AMP.

Iii-B State Evolution

The tap coefficients in the CAMP have been determined so as to guarantee the asymptotic Gaussianity of the estimation errors. The author [12] proposed a general error model and used state evolution (SE) to prove that the estimation error before thresholding is asymptotically Gaussian-distributed in the general error model. To prove the asymptotic Gaussianity of the estimation error $h_{t} = x_{t} + A^{T} z_{t} - x$ before the thresholding $f_{t}$ , thus, it is sufficient to show that the error model of the CAMP is included into the general error model.

Let $q_{t + 1} = f_{t} (x + h_{t}) - x$ denote the estimation error after the thresholding. According to the definition of the general error model [12], define $b_{t} = V^{T} {~ q}_{t}$ , $m_{t} = V^{T} h_{t}$ , and

{~ q}_{0} = q_{0}, {~ q}_{t} = q_{t} - ξ_{t - 1} h_{t - 1}

(9)

for $t > 0$ , where $ξ_{t}$ is an abbreviation of $ξ_{t}^{(t)}$ given in (5). Then, $m_{t}$ satisfies the following equation:

	$m_{t} =$	$(I_{N} - Λ) (b_{t} + ξ_{t - 1} m_{t - 1}) + Σ^{T} U^{T} w$			(10)
		$+ t - 1 \sum τ = 0 ξ_{τ}^{(t - 1)} g_{t - τ - 1}^{(1)} (m_{τ} - b_{τ} - ξ_{τ - 1} m_{τ - 1}),$			(10)

with $m_{t} = 0$ for all $t < 0$ .

Proof:

From the definitions of $m_{t}$ and $h_{t}$ , we use the SVD $A = U Σ V^{T}$ to have

m_{t} = V^{T} q_{t} + Σ^{T} U^{T} z_{t} .

(11)

Left-multiplying (4) by $Σ^{T} U^{T}$ and substituting (1) and (11), we obtain

	$m_{t} =$	$(I_{N} - Λ) V^{T} q_{t} + Σ^{T} U^{T} w$			(12)
		$+ t - 1 \sum τ = 0 ξ_{τ}^{(t - 1)} g_{t - τ - 1}^{(1)} (m_{τ} - V^{T} q_{τ}) .$			(12)

Using (9) and the definitions of $b_{t}$ and $m_{t}$ , we arrive at (10).

For $τ = 0, 1, \dots$ and $τ^{'} = 0, \dots, τ$ , define

g_{τ^{'}, τ}^{(k)} = \frac{1}{N} N \sum n = 1 \frac{\partial [Λ^{k} m_{τ}]_{n}}{\partial [b_{τ^{'}}]_{n}} .

(13)

When $g_{τ^{'}, τ}^{(0)} = 0$ holds for all $τ^{'}$ and $τ$ , the general error model in [12] includes the error model of the CAMP. The following theorem implies that the inclusion is correct in the large system limit. Thus, the asymptotic Gaussianity of the estimation errors is guaranteed in the CAMP.

Theorem 1

For all $τ = 0, 1, \dots$ and $τ^{'} = 0, \dots, τ$ , the almost sure convergence $g_{τ^{'}, τ}^{(0)} a . s . \to 0$ holds in the large system limit.

Proof:

The proof is by induction to show

$g_{τ^{'}, τ}^{(0)} a . s . \to 0$ ,
the almost sure convergence of $ξ_{τ}$ to a constant,
Let ${~ g}_{0}^{(k)} = g_{τ, τ}^{(k)}$ and ${~ g}_{τ^{'}, τ}^{(k)} = g_{τ^{'}, τ}^{(k)} / ξ_{τ^{'}}^{(τ - 1)}$ for $τ^{'} < τ$ . ${~ g}_{τ^{'}, τ}^{(k)}$ depends on $τ$ and $τ^{'}$ only through $τ - τ^{'}$ .

According to [12, Theorem 1], the statement 2) follows from the statement 1). Thus, we only focus on the first and last statements. For $τ = 0$ , we use (10) to obtain $g_{0, 0}^{(0)} = μ_{0} - μ_{1} a . s . \to 0$ , because of $μ_{0} = 1$ and $μ_{1} a . s . \to 1$ .

For some $t$ , assume the three statements for all $τ < t$ and $τ^{'} \leq τ$ . We shall prove the first and last statements for $τ = t$ .

We first prove the statement 3). For $t^{'} = t$ and $t^{'} = t - 1$ , we use (10) to obtain

g_{t, t}^{(k)} = μ_{k} - μ_{k + 1},

(14)

g_{t - 1, t}^{(k)} = ξ_{t - 1} (g_{t - 1, t - 1}^{(k)} - g_{t - 1, t - 1}^{(k + 1)}) + ξ_{t - 1} g_{0}^{(1)} (g_{t - 1, t - 1}^{(k)} - μ_{k}),

(15)

where we have used the second induction hypothesis. Similarly, for $t^{'} \leq t - 2$ we have

	$g_{t^{'}, t}^{(k)} =$	$ξ_{t - 1} (g_{t^{'}, t - 1}^{(k)} - g_{t^{'}, t - 1}^{(k + 1)}) + t - 1 \sum τ = t^{'} ξ_{τ}^{(t - 1)} g_{t - τ - 1}^{(1)} g_{t^{'}, τ}^{(k)}$
	$-$	$ξ_{t^{'}}^{(t - 1)} g_{t - t^{'} - 1}^{(1)} μ_{k} - t - 1 \sum τ = t^{'} + 1 ξ_{τ - 1}^{(t - 1)} g_{t - τ - 1}^{(1)} g_{t^{'}, τ - 1}^{(k)} .$			(16)

From the last induction hypothesis, we can define ${~ g}_{τ - τ^{'}}^{(k)} = g_{τ^{'}, τ}^{(k)} / ξ_{τ^{'}}^{τ - 1}$ for $τ < t$ . Using (14) and this change of variables yields

{~ g}_{t - 1, t}^{(k)} = {~ g}_{0}^{(k)} - {~ g}_{0}^{(k + 1)} - g_{0}^{(1)} μ_{k + 1},

(17)

	${~ g}_{t^{'}, t}^{(k)} =$	${~ g}_{t - t^{'} - 1}^{(k)} - {~ g}_{t - t^{'} - 1}^{(k + 1)} + t - t^{'} - 1 \sum τ = 1 g_{t - t^{'} - τ - 1}^{(1)} ({~ g}_{τ}^{(k)} - {~ g}_{τ - 1}^{(k)})$			(18)
		$- g_{t - t^{'} - 1}^{(1)} μ_{k + 1}$			(18)

for $t^{'} \leq t - 2$ , with ${~ g}_{0}^{(k)} = g_{t, t}^{(k)}$ . Since the right-hand sides (RHSs) depend on $t^{'}$ and $t$ only through $t - t^{'}$ , we find that the statement 3) holds for $τ = t$ , and can re-write the left-hand sides of (17) and (18) as ${~ g}_{1}^{(k)}$ and ${~ g}_{t - t^{'}}^{(k)}$ , respectively.

Finally, we prove the statement 1). It is sufficient to prove ${~ g}_{t^{'}}^{(0)} a . s . \to 0$ and ${~ g}_{t^{'}}^{(k)} + g_{t^{'}}^{(k)} a . s . \to 0$ for all $t^{'} = 0, \dots, t$ and $k$ . The proof is by induction. For $t^{'} = 0$ , we have ${~ g}_{0}^{(0)} = μ_{0} - μ_{1} a . s . \to 0$ . Comparing (6) and ${~ g}_{0}^{(k)} = μ_{k} - μ_{k + 1}$ yields ${~ g}_{0}^{(k)} = - g_{0}^{(k)}$ .

For $t^{'} = 1$ , we use (17), ${~ g}_{0}^{(0)} a . s . \to 0$ , and ${~ g}_{0}^{(1)} = - g_{0}^{(1)}$ to obtain ${~ g}_{1}^{(0)} a . s . = - {~ g}_{0}^{(1)} - g_{0}^{(1)} + o (1) a . s . \to 0$ . Furthermore, we use (7), (17), and ${~ g}_{0}^{(k)} = - g_{0}^{(k)}$ to find ${~ g}_{1}^{(k)} + g_{1}^{(k)} a . s . \to 0$ .

Assume ${~ g}_{t^{'}}^{(0)} a . s . \to 0$ and ${~ g}_{t^{'}}^{(k)} + g_{t^{'}}^{(k)} a . s . \to 0$ for all $t^{'} < τ \in {2, \dots, t}$ , and prove ${~ g}_{τ}^{(0)} a . s . \to 0$ and ${~ g}_{τ}^{(k)} + g_{τ}^{(k)} a . s . \to 0$ . For the former statement, we use (18) and the induction hypotheses ${~ g}_{t^{'}}^{(0)} a . s . \to 0$ and ${~ g}_{τ - 1}^{(1)} + g_{τ - 1}^{(1)} a . s . \to 0$ to obtain

{~ g}_{τ}^{(0)} a . s . = - {~ g}_{τ - 1}^{(1)} - g_{τ - 1}^{(1)} + o (1) a . s . \to 0.

(19)

For the latter statement, we use (8), (18), and the induction hypothesis ${~ g}_{t^{'}}^{(k)} + g_{t^{'}}^{(k)} a . s . \to 0$ to find ${~ g}_{τ}^{(k)} + g_{τ}^{(k)} a . s . \to 0$ . Thus, ${~ g}_{t^{'}}^{(0)} a . s . \to 0$ and ${~ g}_{t^{'}}^{(k)} + g_{t^{'}}^{(k)} a . s . \to 0$ hold for all $t^{'} = 0, \dots, t$ and $k$ . In other words, we have proved the statement 1).

Iii-C Closed-Form Solution

The sequence ${g_{t}^{(k)}}$ may be computed by solving the discrete-time dynamical systems (6)–(8) numerically when the moment sequence ${μ_{k}}$ is given. However, it is possible to obtain a closed-form solution of the tap coefficients ${g_{t}^{(1)}}$ via the $η$ -transform $η (z)$ of the asymptotic eigenvalue distribution of $A^{T} A$ [14], given by

η (z) = lim M = δ N \to \infty \infty \sum k = 0 μ_{k} (- z)^{k} .

(20)

Theorem 2

Let $G_{k} (y)$ denote the generating function of ${g_{t}^{(k)}}$ with respect to $t = 0, 1 \dots$ , defined as

G_{k} (y) = \infty \sum t = 0 y^{t} g_{t}^{(k)} .

(21)

Then, $G_{1} (y)$ is implicitly given by $η (x_{s}) = 1 - y$ in the large system limit, with

x_{s} = \frac{y}{(1 - y) {1 - y G_{1} (y)}} .

(22)

Proof:

Define the generating function of ${g_{t}^{(k)}}$ as

G (x, y) = \infty \sum k = 0 x^{k} G_{k} (y) .

(23)

Theorem 2 follows from the following closed-form expression of $G (x, y)$ :

G (x, y) a . s . = \frac{x η (- x) {y G_{1} (y) - 1} + η (- x) - 1}{(1 - y) {1 - y G_{1} (y)} x + y} + o (1) .

(24)

By definition, $G (x, y)$ is a polynomial of $x$ and $y$ . Thus, the numerator of (24) must be zero when the denominator is zero. The point $- x_{s}$ given in (22) is a zero of the denominator for any $y$ . Thus, we let the numerator at $x = - x_{s}$ be zero to obtain $η (x_{s}) = 1 - y$ . Thus, we arrive at Theorem 2.

To complete the proof of Theorem 2, we shall prove (24). We first derive a closed-form expression of (21), given by

G_{k} (y) = g_{0}^{(k)} + g_{1}^{(k)} y + \infty \sum t = 2 y^{t} g_{t}^{(k)} .

(25)

Substituting (8) into the last term on the RHS of (25) yields

	$\infty \sum t = 2 y^{t} g_{t}^{(k)} = \infty \sum t = 2 y^{t} g_{t - 1}^{(k)} - \infty \sum t = 2 y^{t} g_{t - 1}^{(k + 1)} + \infty \sum t = 2 y^{t} g_{t - 1}^{(1)} μ_{k + 1}$
	$+ \infty \sum t = 2 y^{t} t - 1 \sum τ = 1 g_{t - τ - 1}^{(1)} g_{τ}^{(k)} - \infty \sum t = 2 y^{t} t - 1 \sum τ = 1 g_{t - τ - 1}^{(1)} g_{τ - 1}^{(k)} .$		(26)

For the first three terms, we have

\infty \sum t = 2 y^{t} g_{t - 1}^{(k^{'})} = y \infty \sum t = 1 y^{t} g_{t}^{(k^{'})} = y G_{k^{'}} (y) - g_{0}^{(k^{'})} y

(27)

for $k^{'} = 1, k, k + 1$

. Since the Z-transform of convolution is the product of Z-transforms, the last term reduces to

	$\infty \sum t = 2 y^{t} t - 1 \sum τ = 1 g_{t - τ - 1}^{(1)} g_{τ - 1}^{(k)} =$	$y^{2} \infty \sum t = 0 y^{t} t \sum τ = 0 g_{t - τ}^{(1)} g_{τ}^{(k)}$
	$=$	$y^{2} G_{1} (y) G_{k} (y) .$			(28)

Similarly, for the fourth term we have

	$\infty \sum t = 2 y^{t} t - 1 \sum τ = 1 g_{t - τ - 1}^{(1)} g_{τ}^{(k)} =$	$\infty \sum t = 1 y^{t + 1} t \sum τ = 1 g_{t - τ}^{(1)} g_{τ}^{(k)}$
	$=$	$y G_{1} (y) {G_{k} (y) - g_{0}^{(k)}} .$			(29)

Using these results, as well as (6) and (7), we obtain the closed-form expression

G_{k} (y) = \frac{μ_{k} y G_{1} (y) - y G_{k + 1} (y) + g_{0}^{(k)}}{(1 - y) {1 - y G_{1} (y)}} .

(30)

The closed-form expression (24) follows from (30). Using the $η$ -transform (20) yields

G (x, y) a . s . = \frac{η (- x) y G_{1} (y) - x^{- 1} y G (x, y) + G (x, 0)}{(1 - y) {1 - y G_{1} (y)}} + o (1),

(31)

where we have used $G_{0} (y) a . s . \to 0$ obtained from Theorem 1. Applying $G (x, 0) = x^{- 1} {η (- x) - 1} - η (- x)$ obtained from (6) and solving $G (x, y)$ , we arrive at (24).

The following corollary implies that the CAMP reduces to conventional AMP when the sensing matrix has i.i.d. Gaussian elements with mean proportional to $M^{- 1 / 2}$ . Thus, the CAMP has no ability to handle this non-zero mean case.

Corollary 1

If $A$ has independent Gaussian elements with mean $\sqrt{γ / M}$ and variance $(1 - γ) / M$ for any $γ \in [0, 1)$ , the CAMP is equivalent to conventional AMP.

Proof:

The R-transform $R (z)$ [14, Section 2.4.2] of the asymptotic eigenvalue distribution of $A^{T} A$ is given by

R (z) = \frac{δ}{δ - z} .

(32)

Using Theorem 2 and the following relationship between the $η$ and R transforms:

η (z) = \frac{1}{1 + z R (- z η (z))},

(33)

we obtain

1 - y = \frac{1}{1 + δ x_{s} {δ + x_{s} (1 - y)}^{- 1}},

(34)

where $x_{s}$ is given by (22). Substituting (22) and solving $G_{1} (y)$ , we arrive at $G_{1} (y) = δ^{- 1}$ .

From the definition (21), we find $g_{0}^{(1)} = δ^{- 1}$ and $g_{t}^{(1)} = 0$ for all $t > 0$ . This implies that the update rule (4) reduces to that corresponding to conventional AMP.

The following corollary is utilized in numerical simulations.

Corollary 2

If $A$ is orthogonally invariant and has non-zero singular values $σ_{0} \geq \dots \geq σ_{M - 1} > 0$ satisfying condition number $κ = σ_{0} / σ_{M - 1} \geq 1$ , $σ_{m} / σ_{m - 1} = κ^{- 1 / (M - 1)}$ , and $σ_{0}^{2} = N (1 - κ^{- 2 / (M - 1)}) / (1 - κ^{- 2 M / (M - 1)})$ , then $g_{t}^{(1)} = g_{t} + C / (κ^{2} - 1)$ holds for all $t$ , with

g_{t} = t - 1 \sum τ = 0 h_{t - τ} g_{τ} - h_{t + 1}, g_{0} = - h_{1},

(35)

h_{t} = \frac{C^{t - 1}}{t!} - \frac{C^{t}}{(t + 1)!}, C = \frac{2}{δ} ln κ .

(36)

Proof:

Since $μ_{k} = N^{- 1} σ_{0}^{2 k} (1 - κ^{- 2 k M / (M - 1)}) / (1 - κ^{- 2 k / (M - 1)})$ holds for all $k > 0$ , we use (20) and $N (1 - κ^{- a / (M - 1)}) \to δ^{- 1} a ln κ$ for any $a \in R$ to find

	$η (z) =$	$1 + \infty \sum k = 1 (- z)^{k} {\frac{C}{(1 - κ^{- 2})}}^{k} \frac{(1 - κ^{- 2 k})}{k C}$
	$=$	$1 - \frac{1}{C} ln {\frac{δ (κ^{2} - 1) + 2 κ^{2} z ln κ}{δ (κ^{2} - 1) + 2 z ln κ}},$			(37)

where the second equality follows from $ln (1 + x) = \sum_{k = 1}^{\infty} (- 1)^{k - 1} k^{- 1} x^{k}$ for all $| x | < 1$ . Using Theorem 2 yields

G_{1} (y) = \frac{C}{(κ^{2} - 1) (1 - y)} + \frac{1}{y} + \frac{C}{(1 - y) (1 - e^{C y})} .

(38)

It is an exercise to confirm that the generating function of $g_{t}$ in Corollary 2 is equal to the sum of the second and last terms. Thus, Corollary 2 holds.

Iv Numerical Simulation

The CAMP is compared to AMP and OAMP/VAMP in terms of the mean-square error (MSE) in signal recovery. As an example of ill-conditioned sensing matrices in Corollary 2, $A = d i a g {σ_{0}, \dots, σ_{M - 1}} H$ is considered for $M \leq N$ , with $σ_{m}$ denoting the $m$ th singular value in Corollary 2. The $M$ rows of $H \in R^{M \times N}$ are selected uniformly and randomly from the rows of the $N \times N$ Hadamard orthogonal matrix.

We assume the Bernoulli-Gaussian (BG) prior: Each signal element takes $0$

with probability

1 - ρ

. Otherwise, it is sampled from the zero-mean Gaussian distribution with variance

ρ^{- 1}

. We use the soft thresholding [1]

f_{t} (x) = ⎧ ⎪ ⎨ ⎪ ⎩ \begin{matrix} x - θ_{t} & for x \geq θ_{t}, 0 & for x \in (- θ_{t}, θ_{t}), x + θ_{t} & for x \leq - θ_{t} . \end{matrix}

(39)

For the sensing matrix in Corollary 2, we have no SE results of the CAMP or AMP for designing the threshold $θ_{t}$ . Thus, the threshold $θ_{t}$ is fixed to a constant $θ$ over all iterations, which was optimized via an exhaustive search.

[width=]fig1.eps

Fig. 1: MSE versus the condition number

σ_{0} / σ_{M - 1}

for signal density

ρ = 0.1

100

iterations,

M = 614

N = 2^{10}

, and

1 / σ^{2} = 30

dB.

Figure 1 shows the MSEs of the CAMP, AMP, and OAMP/VAMP estimated from $10^{5}$ independent trials. The CAMP outperforms AMP and achieves the MSEs comparable to OAMP/VAMP. The inferior performance of AMP is due to a bad convergence property of AMP. Using a large threshold $θ$ improves the convergence property. Exhaustive search of $θ$ implied that larger thresholds are required for AMP to converge than for the other algorithms. Thus, we conclude that CAMP improves the convergence property of AMP.

References

[1] D. L. Donoho, A. Maleki, and A. Montanari, “Message-passing algorithms for compressed sensing,” Proc. Nat. Acad. Sci., vol. 106, no. 45, pp. 18 914–18 919, Nov. 2009.
[2] D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289–1306, Apr. 2006.
[3] E. J. Candés, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489–509, Feb. 2006.
[4] M. Bayati and A. Montanari, “The dynamics of message passing on dense graphs, with applications to compressed sensing,” IEEE Trans. Inf. Theory, vol. 57, no. 2, pp. 764–785, Feb. 2011.
[5]
M. Bayati, M. Lelarge, and A. Montanari, “Universality in polytope phase transitions and message passing algorithms,”
Ann. Appl. Probab., vol. 25, no. 2, pp. 753–822, Apr. 2015.
[6] S. Rangan, P. Schniter, A. Fletcher, and S. Sarkar, “On the convergence of approximate message passing with arbitrary matrices,” IEEE Trans. Inf. Theory, vol. 65, no. 9, pp. 5339–5351, Sep. 2019.
[7] F. Caltagirone, L. Zdeborová, and F. Krzakala, “On convergence of approximate message passing,” in Proc. 2014 IEEE Int. Symp. Inf. Theory, Honolulu, HI, USA, Jul. 2014, pp. 1812–1816.
[8] J. Ma and L. Ping, “Orthogonal AMP,” IEEE Access, vol. 5, pp. 2020–2033, Jan. 2017.
[9] S. Rangan, P. Schniter, and A. K. Fletcher, “Vector approximate message passing,” IEEE Trans. Inf. Theory, vol. 65, no. 10, pp. 6664–6684, Oct. 2019.
[10] M. Opper and O. Winther, “Expectation consistent approximate inference,” J. Mach. Learn. Res., vol. 6, pp. 2177–2204, Dec. 2005.
[11] K. Takeuchi, “Rigorous dynamics of expectation-propagation-based signal recovery from unitarily invariant measurements,” IEEE Trans. Inf. Theory, vol. 66, no. 1, pp. 368–386, Jan. 2020.
[12] ——, “A unified framework of state evolution for message-passing algorithms,” in Proc. 2019 IEEE Int. Symp. Inf. Theory, Paris, France, Jul. 2019, pp. 151–155.
[13] F. Hiai and D. Petz, The Semicircle Law, Free Random Variables, and Entropy. Amer. Math. Soc., 2000.
[14] A. M. Tulino and S. Verdú, Random Matrix Theory and Wireless Communications. Hanover, MA USA: Now Publishers Inc., 2004.

Convolutional Approximate Message-Passing

Authors

Bayes-Optimal Convolutional AMP

On the Convergence of Orthogonal/Vector AMP: Long-Memory Message-Passing Strategy

Precoding via Approximate Message Passing with Instantaneous Signal Constraints

Universality of Linearized Message Passing for Phase Retrieval with Structured Sensing Matrices

Sparse Bayesian Learning Using Approximate Message Passing with Unitary Transformation

A Low Complexity MAP Detector for OTFS Modulation in Logarithmic Domain

Approximate Message Passing with Restricted Boltzmann Machine Priors

I Introduction

Ii Measurement Model

Iii Convolutional AMP

Iii-a Algorithm

Iii-B State Evolution

Proof:

Theorem 1

Proof:

Iii-C Closed-Form Solution

Theorem 2

Proof:

Corollary 1

Proof:

Corollary 2

Proof:

Iv Numerical Simulation

References

Convolutional Approximate Message-Passing

Authors

Related Research

Bayes-Optimal Convolutional AMP

On the Convergence of Orthogonal/Vector AMP: Long-Memory Message-Passing Strategy

Precoding via Approximate Message Passing with Instantaneous Signal Constraints

Universality of Linearized Message Passing for Phase Retrieval with Structured Sensing Matrices

Sparse Bayesian Learning Using Approximate Message Passing with Unitary Transformation

A Low Complexity MAP Detector for OTFS Modulation in Logarithmic Domain

Approximate Message Passing with Restricted Boltzmann Machine Priors

This week in AI

I Introduction

Ii Measurement Model

Iii Convolutional AMP

Iii-a Algorithm

Iii-B State Evolution

Proof:

Theorem 1

Proof:

Iii-C Closed-Form Solution

Theorem 2

Proof:

Corollary 1

Proof:

Corollary 2

Proof:

Iv Numerical Simulation

References