The high-order block RIP for non-convex block-sparse compressed sensing

06/11/2020
by   Jianwen Huang, et al.
Southwest University
0

This paper concentrates on the recovery of block-sparse signals, which is not only sparse but also nonzero elements are arrayed into some blocks (clusters) rather than being arbitrary distributed all over the vector, from linear measurements. We establish high-order sufficient conditions based on block RIP to ensure the exact recovery of every block s-sparse signal in the noiseless case via mixed l_2/l_p minimization method, and the stable and robust recovery in the case that signals are not accurately block-sparse in the presence of noise. Additionally, a lower bound on necessary number of random Gaussian measurements is gained for the condition to be true with overwhelming probability. Furthermore, the numerical experiments conducted demonstrate the performance of the proposed algorithm.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/11/2020

The block mutual coherence property condition for signal recovery

Compressed sensing shows that a sparse signal can stably be recovered fr...
08/18/2020

A Two Stage Generalized Block Orthogonal Matching Pursuit (TSGBOMP) Algorithm

Recovery of an unknown sparse signal from a few of its projections is th...
09/09/2019

Sparse linear regression with compressed and low-precision data via concave quadratic programming

We consider the problem of the recovery of a k-sparse vector from compre...
10/08/2021

A Framework for Private Communication with Secret Block Structure

Harnessing a block-sparse prior to recover signals through underdetermin...
11/10/2011

Accuracy guaranties for ℓ_1 recovery of block-sparse signals

We introduce a general framework to handle structured models (sparse and...
10/13/2021

The springback penalty for robust signal recovery

We propose a new penalty, named as the springback penalty, for construct...
04/04/2011

Block-Sparse Recovery via Convex Optimization

Given a dictionary that consists of multiple blocks and a signal that li...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Block-sparse signal recovery (BSR) appears in some fields of sparse modelling and machine learning, including color imaging

[13], equalization of sparse communication channels [2]

, multi-response linear regression

[3] and imagine annotation [4] and so forth. Essentially, the important problem in BSR is how to reconstruct a block-sparse or approximately block-sparse signal from a linear system. Commonly, one thinks over the below model:

where is the observation measurement, is a known measurement matrix (or sensing matrix) with , and is a vector of measurement errors. Generally, the conventional compressed sensing (CS) simply thinks out the sparsity of the signal to be recovered, however it doesn’t consider any additional structure, i.e., non-zero elements appear in blocks (or clusters) rather than being arbitrarily spread all over the vector. We call these signals as the block-sparse signals. In order to define block sparsity, we need to give several additional notations. Suppose that the signal over the block index set , then we can describe the signal as

(1.1)

where represents the th block of and . We call a vector as block -sparse over if is non-zero for no more than indices [5]. In fact, if the block-sparse structure of signal is neglected, the conventional CS doesn’t efficiently treat such structured signal. To reconstruct block-sparse signal, researchers [5] [6] proposed the following mixed -minimization:

(1.2)

where is the mixed norm of a vector . The set stands for some noise structure,

(1.3)

and

(1.4)

where represents the conjugate transpose of the matrix . (1.2) is a convex optimization issue and could be converted into a second-order cone program, so can be solved efficiently.

To study the theoretical performance of mixed -minimization, Eldar and Mishali [5] proposed the definition of block restricted isometry property (block RIP).

Definition 1.1.

(Block RIP [5]) Given a matrix , for every block -sparse over , there is a positive number , if

(1.5)

then the matrix obeys the -order block RIP over . Define the block RIP constant (RIC) as the smallest positive constant such that (1.5) holds for all that are block -sparse.

For the reminder of this paper, for simplicity, represents the block RIP constant . Eldar and Mishali [5] showed that the mixed -minimization method can exactly reconstruct any block -sparse signal when the measurement matrix fulfills the block RIP with . Later, Lin and Li [7] improved the sufficient condition on to , and builded condition for accurate recovery. In 2019, the conclusions of literature [8] and [9] together present a complete characterization to the block RIP condition on that the mixed minimization method guarantees the block-sparse signal recovery in the field of block-sparse compressed sensing.

Recently, a lot of researchers [10] [11] [12] have revealed that minimization not only constantly needs less constrained the RIP requirements, but also could ensure exact recovery for smaller compared with the minimization. In the present paper, we are interested in investigating the stable reconstruction of block-sparse signals by the mixed minimization as follows:

(1.6)

where . Simulation experiments [13] [14] indicated that fewer linear measurements are needed for accurate reconstruction when than when . More related work can be found in literature [15] [16] [17] [18] [19] [20] [21].

In this paper, we further investigate the high-order block RIP conditions for the exact and stable reconstruction for (approximate) block-sparse signals by mixed minimization. The crux is extend sparse representation of an -polytope (Lemma 2.2 [22]) to the block scenario. With this technique, we obtain a sufficient condition on RIC that guarantees the exact and stable reconstruction of approximate block-sparse signals via mixed minimization, and establish error bounds between the solution to (1.6) and the signal to be recovered. Obviously, when is accurately block-sparse and (i.e., ), we will derive the accurate reconstruction condition. Particularly, we will determine how many random Gaussian measurements suffice for the condition to hold with high probability.

The remainder of the paper is constructed as follows. In Section 2, we will provide some notations and a few lemmas. In Section 3, we will present the main results, and the associating proofs are given in Section 5. In Section 4, a series of numerical experiments are presented to support our theoretical results. Lastly, the conclusion is drawn in Section 6.

2 Preliminaries

Throughout this article, we use the below notations unless special mentioning. For a subset in , denotes the complement of in . For any vector , represents a vector which is equal to on block indices and displaces other blocks with zero. Denote by block indices of the largest block in norm of the vector , i.e., for any and . We represent as with all but the largest blocks in norm set to zero. Henceforth, we invariably choose that , where is the minimizer of (1.6).

In order to prove our main results, it is necessary to present the below lemma which is a crucial technical tool. Factually, we extend sparse expression of an -polytope proposed by [22] to the block context.

Lemma 2.1.

For a positive integer , a positive number and given , define the block -polytope by

Then any can be expressed as the convex combination of block -sparse vectors, i.e.,

Here and and . In addition,

(2.1)
Proof.

We can prove the assertion holds by induction. If is block -sparse, we can set and , then . Suppose that assertion holds for all block -sparse vectors (). Then for any block -sparse vectors such that and , without loss of generality suppose that is not block -sparse (otherwise the statement is naturally true by assumption of ). Besides, can be represented as , where , is equal to the largest for every , is equal to the next largest , etc. Here denotes a unit vector in , which is equal to on the th largest block of and zero other places. Set , and fix , where . Then we have and

Denote the set

(2.2)

It is easy to see that , hence is not empty. Then we note that , which implies

(2.3)

It follows that

Set

where . Then by simple calculations, we obtain and , where is block -sparse for all . Finally, since is block -sparse, under the induction assumption, we have , where is block -sparse, and , . Hence, , which implies that statement is true for . ∎

We will utilize the below lemma in the process of proving the main conclusions, which is a useful important inequality.

Lemma 2.2.

(Lemma 5.3 [23]) Suppose that , , , then for all ,

More generally, assume that , and , then for all ,

In view of the definition of , and , we get the below lemma.

Lemma 2.3.

Recall that , where is the solution to (1.6). It holds that

Proof.

Assume that is the block index set over blocks with largest norm of the vector . Therefore, . By applying the minimality of the solution and the reverse triangular inequality of , we get

which implies

Note that and . Combining with the above inequalities, the desired result can be derived. ∎

Lemma 2.4.

(Lemma 5.1 [25]) Let

be a random matrix whose entries obey one of the distributions given by (

3.6) and that fulfills the concentration inequality

(2.4)

where the probability is taken over all matrices and is a constant relying merely on and such that for all , . Suppose that . Then, for any , we have

(2.5)

for all -sparse vectors with probability

(2.6)

3 Main Results

Based on the knowledge prepared above, we present the main results in this part-a high-order block RIP condition for the robust reconstruction of arbitrary signals with block structure via mixed minimization. In the case that the signal to be recovered is block-sparse, the condition can respectively guarantee the accurate construction and stable recovery in the noise-free case and in the noise situation. When the original signal is not block-sparse and the linear measurement is corrupted by noise, the below result presents a sufficient condition for recovery of structured signals.

Theorem 3.1.

Let be noisy measurements of a signal with , and . Assume that with in (1.6). If satisfies the block RIP with

(3.1)

for some , where is the sole positive solution of the equation

(3.2)

Then the solution to (1.6) fulfills

(3.3)

where

Remark 3.1.

In the case of , (3.3) is the same as Theorem 2 in [24].

Theorem 3.2.

Let be noisy measurements of a signal with , and . Assume that with in (1.6). If satisfies the block RIP with for some , then the solution to (1.6) fulfills

(3.4)

where

Remark 3.2.

In the case of , we obtain the same results as Theorem 3 in [24].

Corollary 3.1.

Under the same conditions as in Theorem 3.1, suppose that and is block -sparse. Then can be accurately reconstructed via

(3.5)

In the following, we will decide how many random Gaussian measurements are demanded for (3.1) to be fulfilled with overwhelming probability. In this sequel, let be a probability measure space and

be a random variable which follows one of the following probability distributions:

(3.6)

Given and , the random matrices can be produced by making choice of the elements as independent copies of . This generates the random matrices .

Theorem 3.3.

Let be an matrix with whose elements are i.i.d. random variables defined by (3.6). If

the next assertion holds with probability more than
: for any block -sparse signal over with , is the unique solution to (3.5) when the matrix satisfies .

4 Numerical experiments

In this section, we carry out a few numerical simulations to hold out the application of our theoretical results. We could transform the constrained optimization problem (1.6) into an alternative unconstrained form below:

(4.1)

Solving the problem (4.1), we adopt the standard Alternating Direction Method of Multipliers (ADMM)[27][28][26]. Utilizing a auxiliary variable , we can rewrite the formulation (4.1) as

(4.2)

The augmented Lagrangian function of the above problem is

(4.3)

where is a Lagrangian multiplier, and is a penalty parameter. Then, ADMM composes of the below three steps:

(4.4)
(4.5)
(4.6)

The solution of problem (4.4) is explicitly provided by

(4.7)

To use the existing conclusions on the proximity operator of -norm (), by employing the inequality: for and , the optimization problem (4.5) can be converted into

(4.8)

In our experiments, without loss of generality, we think over the block-sparse signal with even block size, i.e., , , and take the signal length . For each experiment, first of all, we randomly produce block-sparse signal

with the amplitude of each nonzero entry generated according to the Gaussian distribution. We use an

orthogonal Gaussian random matrix as the measurement matrix . We set the number of random measurement unless otherwise specified. With and , we generate the linear measurement by , where is the Gaussian noise vector. Each given experimental result is an average over 100 independent trails.

In Fig. 1a, we produce the signals by making choice of blocks uniformly at random with , i.e., the block size . The relative error of recovery is plotted versus the regularization parameter for the different values of , i.e., . The ranges from to . From the figure, the parameter is a proper choice. Fig. 1b presents experimental results regarding the performance of the non-block algorithm and the block algorithm with . Two curves of relative error are provided via mixed minimization and orthogonal greedy algorithm (OGA) [29]. Fig. 1b reveals the signal construction is quite significant in signal recovery.

Figure 1: (a) Recovery performance of mixed minimization versus for block size , (b) Recovery performance of mixed minimization with and OGA and the number of nonzero entries

Signal-to-noise ratio (SNR, SNR) versus the values of and the nonzero entries , the results are respectively given in Fig. 2a and b. In Fig. 2a, the values of vary from to , and in Fig. 2b, the number of nonzero entries ranges from to . Figs. 2a and b evidences that mixed minimization performs better than that of standard minimization. Figs. 2a and b provide the relationship between the relative error and the number of measurements in different block sizes and the values of .

Figure 2: SNR versus the values of and the number of nonzero entries in (a) and (b) respectively (a) For , (b) For .
Figure 3: Relative error versus the number of measurements for (a) , and (b) ,

Eventually, we compare the performance of Group-Lp for with other typical algorithms consisting of Block-OMP algorithm[30], Block-SL0 algorithm [31] for solver and Block-ADM algorithm [32]. We exploit SNR to weigh the algorithm efficiency. In Fig. 4a, we select signals whose block size with the number of nonzero entries as the test signals, and in Fig. 4b, the block size . One can see that, overall, the performance of Group-Lp () is much better than that of other three algorithms.

Figure 4: Comparison recovery performance with respect to SNR (a) Number of nonzero coefficients and (b) block size

5 The proofs of Main Results

Proof of Theorem 3.1. First of all, suppose that is an integer. Recollect that , where is the solution to (1.6) with . Then, by employing the condition of Theorem 3.1, we have

that is,

Denote . Then, by utilizing Lemma 2.3, we have

(5.1)

Through applying Lemma 2.1 and combining with (5), we have , where is block -sparse, with , and

(5.2)

Hence,

(5.3)

where (a) follows from Hlder’s inequality, (b) is due to the fact that , and given , and (c) is from the fact that , .

From Cauchy-Schwartz inequality and the definition of block RIP, we get

(5.4)

Since

(5.5)

therefore

(5.6)

Take . Then,

(5.7)

Furthermore, both and are block -sparse, because is block -sparse, and is block -sparse.

Observe that the identity below [22]

(5.8)

where .

First of all, we determine the left hand side (LHS) of (5). Putting (5.7) into LHS of (5) and combining with (5.6) and the concept of block RIP, we get