I Introduction
THE last decade has seen the burgeoning development of Compressed Sensing (CS) [1, 2] and its wide-spread applications in many fields. At the core of CS is the problem of efficiently recovering a sparse signal from a relatively small number of linear measurements. Mathematically, for any given signal , we say that it is sparse if most of its entries are zero. More specifically, if it has at most non-zero entries, i.e., , we call it a -sparse signal. In standard CS, one usually observes the linear measurements of the sparse signal via , where () is a given measurement matrix. To recover such a sparse signal, a natural idea is to search the sparsest solution among all the possible solutions. This directly leads to the following -norm minimization problem
(1) |
Unfortunately, this problem is NP-hard in general, and hence it is computationally infeasible. Instead, some algorithms which aims to pursue the suboptimal solutions of (1) were proposed, see, e.g., [3, 4, 5] and their variants [6, 7, 8, 9]. Importantly, many of these algorithms have been proved to perform well under certain conditions.
Besides the above algorithm strategies, there also exist many other efficient approaches [10, 11, 12, 13, 14, 15] which can circumvent the NP-hardness of (1), and a popular one is the constrained -norm minimization method which solves
(2) |
where is the
-norm of vector. Problem (
2) is convex and therefore can be well addressed by many convex optimization softwares. To theoretically investigate the equivalence between (1) and (2), one often adopts the Restricted Isometry Constant (RIC) of matrix with order, denoted by , which is defined to be the smallest value of such tahtfor every -sparse vector . This notation was first proposed by Candès and Tao in [10], where they have shown that (2) is equivalent to (1) in noiselessly recovering any -sparse signals when . Subsequently many researchers were committed to improving this condition, see, e.g., [16, 17, 18, 19, 20, 21, 22, 23]. In more general application scenarios of CS, one often wishes to recover the original signal (may not be exactly sparse) from the noisy observation with
(3) |
where is the unknown measurement noise/error, which directly leads to the following optimization problem
(4) |
where . Obviously, (4) will reduces to (2) if one takes . It should also be noted that the above-mentioned exact recovery conditions are still available to guarantee the robust recovery of signals from (4) in the presence of noise.
On the other hand, when used in many practical applications, particularly the applications where the input data are in large scale, the constrained problem (4) is not always convenient to solve. Instead, one often solves its unconstrained counterpart, i.e., the (-norm) regularized -norm minimization problem
(5) |
where is the nonnegative tradeoff parameter. This problem is also known as the Lasso estimator [24] or the Basis Pursuit DeNoising (BPDN) [25] , and it can also be solved efficiently by many algorithms, see, e.g., [26, 27, 28, 29, 30]. Recently, the relation between (4) and (5) was carefully investigated by Zhang, Yuan and Yin [31] in the context of the non-uniform recovery [32], i.e., the recovery of some specific sparse signals, for example, the sparse signals limited in a specific support. However, when it comes to the uniform recovery [31, 32], i.e., recovering all the (general) sparse signals 111In these signals, both the number and support of their non-zero entries are not known in advance., it is generally believed that (4) and (5) are not exact equivalents. As early as 2008, Zhu [33] has derived the RIC-based theoretical guarantee for (5), which states that one can robustly recover any -sparse signal through (3) with by using (5) under certain , if obeys . However, this work was relatively rarely noted by the researchers. In 2009, Bickel, Ritov and Tsybakov [34] established a RIC-like guarantee for (5) in the presence of the random noise. Recently, some new RIC conditions were obtained to ensure the robust recovery of some unconstrained -analysis approaches under the Dantzig Selector (DS) type noise/error (i.e., ), see [35, 36, 37] for details.
Some random matrices represented by sub-Gaussian random matrices are proved to have a small RIC with overwhelmingly high probability
[32]. However, when used in practical scenarios they often suffer from storage and computation limitations. Moreover, it is also NP-hard in general to find the RIC of any given matrix. To overcome these difficulties, some researchers proposed to reuse the mutual coherence, another powerful tool introduced by Mallat and Zhang [38] in their initial research on matching pursuit. In this paper, denoting by the th column of matrix , we shall define the mutual coherence of matrix aswhere we assume that obeys for . Many deterministic measurement matrices in fact are designed according to the mutual coherence. There are many coherence-based theoretical results for (4), see, e.g., [39, 40, 41, 42, 43, 44]. In particular in [44], Cai, Wang and Xu have shown that any signals (not necessary to be -sparse) can be robustly recovered using (4), if satisfies , and this condition is also sharp for the noiseless recovery of any -sparse signals through (2).
As far as we know, the first (mutual) coherence-based result on (5) was given by Fuchs [41] in 2004 under the non-uniform recovery setting, which states that any fixed signal with non-zero entries (i.e., ) can be uniquely recovered from using (5), if satisfies and in (5) has been taken small enough. Later, Fuchs [45] further investigated (5) for the noisy signal recovery, and shown that if is observed through (3) with and , satisfies for certain , and is assumed to be the optimal solution of (5) under certain (related to , and ), then the support of will be either identical to, or contained in, that of . Moreover, Fuchs also shown that if similar constraints are imposed on , and , and will have their non-zero entries at the same support and with the same signs. Subsequently, Tropp [46] further extended the results in [45] to more general case. In 2010, Ben-Haim, Eldar and Elad [47] revisited (5) under the random noise, and their obtained coherence results have been proved to be better than those induced in [34]. Note that the above coherence results apply only to deal with the signals whose sparsity is known in advance. Recently, using the cumulative coherence [48] tool, Li and Chen [49] established a new uniform recovery condition for (5) to deal with the signal recovery in the presence of noise. Their results shown that if obeys , one can robustly recover any signals corrupted with the DS type noise. However, the noise they considered is based on the DS type noise rather than the often used -norm bounded noise, and the recovery condition they obtained still has much room to improve.
In this paper, equipped with the powerful mutual coherence tool, we investigate the performance guarantees of (5) and its some variants. In summary, the contributions of this paper are listed as follows:
-
We establish a tight uniform recovery condition and two relatively tight error estimates for (5), which are sufficient to guarantee the robust recovery of signals corrupted with the -norm bounded noise.
-
We extend the obtained theoretical results to guarantee the robust recovery of the signals corrupted with the DS type noise and also the structured block-sparse signal recovery in the presence of the bounded noise. To the best of our knowledge, these extended results are established for the first time under the uniform recovery setting.
Ii Notations and Preliminaries
Ii-a Notations
Throughout this paper, we denote for any given integer , and for any given index set . We also denote as a vector whose entries for and 0 otherwise, and where represents certain norm or quasi-norm. For any signal , we denote its best -term approximate as , which is defined as
Besides, for the simplicity of symbol expression we introduce the following two functions
Ii-B Three key lemmas
The proof of our main results heavily relies on the following three lemmas. We start with introducing the first lemma, which provides a RIC-like coherence result for any given matrix.
Lemma 1 ([46, 42]).
Assume that the matrix is standardized to have unit -norm. Then it holds that
(6) |
for all -sparse signals .
Lemma 2.
Proof:
Lemma 3.
If the matrix is standardized to have unit -norm, and obeys
(12) |
for certain integer , then for any vector and any subset with , it holds that
(13) |
where
Remark 1.
It is easy to know from Lemma 3 that both and are two monotone increasing functions on variable . Therefore if one restricts , it will be clear that
(14) |
Iii Main Results
With preparations above, we now present our main results.
Theorem 1.
Remark 2.
Theorem 1 shows that one can robustly recover any signals (may not be -sparse) corrupted with the -norm bounded noise, if the measurement matrix satisfies (17). To the best of our knowledge, we first extend this sharp uniform recovery condition 222The sharp condition/bound here and throughout will refer to . It has been shown in [44] that, any -sparse signal , without exception, can be exactly recovered from (2) under this sharp condition, and there exists a matrix with obeying , and two nonzero -sparse vectors and with disjoint supports such that . derived by Cai, Wang and Xu in [44] for the constrained problem (2) to its unconstrained counterpart, i.e., the unconstrained problem (5). Similar to [36, 37], if we associate with , e.g., setting , then we get a special case of Theorem 1, and one can find this result in Corollary 1. In Remark 3, we will analyze the tightness of these two error estimates under the setting of . Besides, it is also very easy to induce some other special cases of Theorem 1 to cope with several different sparse recovery tasks. For examples, one can consider the robust recovery of any exactly sparse signals, i.e., setting the original signals to be exactly -sparse. The detailed analysis of these cases will become very similar to that of Corollary 1.
Corollary 1.
Remark 3.
Due to the existence of and , the coefficients are not convenient to be analyzed. Fortunately, based on the previous estimates for and , i.e., (14), we can give a rough but simple estimate for each . Specifically,
These upper bound estimates of coefficients make our recovery error, denoted by RE, have the form of
(19) |
where RE stands for or , and and depend only on the value of , which characterizes the gap between the coherence of the selected measurement matrix and its sharp bound. This result also coincides with the ones obtained in [35, 36, 37, 49] for (5) in form. However, one should note that the authors in these literature focus on sparse recovery corrupted with the DS type noise, which is totally different from ours. Despite this, our obtained upper bound estimates to some degree are still better than theirs since a much tighter (or sharp) recovery condition is used. What’s more, some coefficients in these estimates can be further improved if one optimizes some inequalities used to prove Theorem 1.
Proof:
We first assume that for certain integer and denote
then using Lemma 2, Lemma 3, we have
(20) |
where we used . We can known from the condition (17) that
Therefore we can further write (III) as
which implies that
This completes (18). Based on (18), (8) and (13), we have
(21) |
Besides, using (III) together with (8) and (18) again, we can estimate as
(22) |
On the other hand, let be the index set of the largest entries of . Then we know from [36] that
(23) |
Similarly, using Lemma 3 again on index , we also have
This, together with (18) and (III), directly leads to
(24) |
Now, combining (III), (22) and (III), we can estimate as follows:
which completes the proof. ∎
Iv Extensions
In this section, two extensions of Theorem 1 are discussed. They include extending Theorem 1 to guarantee the robust recovery of signals from a DS regularized -norm minimization model in the presence of the DS type noise, and that of the structured block-sparse signals from two regularized mixed -norm minimization models in the presence of the bounded noise. We start with introducing the DS regularized -norm minimization model for signal recovery in the presence of the DS type noise.
Iv-a Robust recovery via a DS regularized -norm minimization
The research on the DS type noise was initiated by Candès and Tao in [50], which aims at recovering the signals corrupted with the DS type noise by solving the following constrained problem
(25) |
Many remarkable results on this problem have been obtained over the past decade, see, e.g., [34, 18, 20, 21, 22, 23, 44]. Similar to the relation of (4) and (5), a closely related problem to (25) is the following DS regularized -norm minimization problem:
(26) |
Inspired by Theorem 1, we also establish a uniform recovery condition and two relatively tight error estimates for (26) to guarantee the robust signal recovery in the presence of such kind of noise, see Theorem 2 for details. This new theorem as well as Theorem 1, to the best of our knowledge, first extends the sharp uniform recovery condition obtained in [44] for (2) to its two unconstrained variants, i.e., (5) and (26), to deal with the signals corrupted with the -norm bounded noise and the DS type noise, respectively. In what follows, we present this theorem.
Theorem 2.
Remark 4.
In general, it is usually suggested to recover the signals corrupted with the DS type noise using the constrained problem (25), see, e.g., [50, 21, 22, 23]. Recently, some researchers proposed to deal with such kind of noise using the unconstrained problem (5), see, e.g., [35, 36, 37, 49], and they also developed a series of recovery conditions and error estimates to realize the robust recovery from (5). However, these results are far from the best. Take for example the mutual coherence condition 333Their original condition in fact is obtained under the cumulative coherence notation. However, whether this condition is sharp was not discussed by the authors. recently obtained in [49], which takes the form of
(28) |
Obviously (28) is rigorously included in our sharp condition (27). In the aspect of algorithm implementation, since (26) is convex, many convex optimization softwares are available to solve it efficiently. Besides, compared to the regularization term (i.e., the second term of the objective function) in (5), the one in (26) is non-smooth and thus non-differentiable. However, if one solves (26) using some non-gradient algorithms, such as the alternating direction method and multipliers [29, 51], (26) is still comparable to (5) in terms of the algorithmic complexity.
Proof:
The proof is very similar to that of Theorem 1, and hence we here only present some technique differences. Our proof also relies on Lemma 1 and the variants of Lemma 2 and Lemma 3. One should keep in mind that the term will be replaced by . Specifically, (7), (8) and (13) are replaced in order by the following inequalities
These, as well as the skills in proving Theorem 1, are sufficient to prove Theorem 2. ∎
Iv-B Structure block-sparse recovery
Our Theorem 1 can still be extended to guarantee the robust recovery of the structured block-sparse signals. Such a kind of signals (data) arise in many applications [52, 53, 54]. We assume w.l.o.g. that there are blocks with block size in signal , and then we can write any signal as
where denotes the th block sub-vector of . If has at most non-zero blocks, i.e., , we refer to such a vector as block -sparse signal. Naturally, a block -sparse signals will reduce the traditional -sparse signal if one takes . Accordingly, we can also write any matrix as
where denotes the th block sub-matrix of . To recover such a structured block-sparse signal, Eldar amd Mishali [55] proposed solving the following mixed -norm minimization problem:
(29) |
where , and they also derived a block-RIC recovery condition for (29). More improved block-RIC conditions can be found in [56, 57, 58, 59]. As early as 2010, Eldar, Kuppinger and Bölcskei [60] have generalized the traditional mutual coherence to the block setting, and show that any block -sparse signal can be exactly recovered via (29) if obeys
(30) |
where and are called block coherence and sub-coherence, respectively, and they are defined as
Obviously, (30) will reduce to (17) if one lets the block sub-matrix be orthonormal 444 This means that , where stands for a identity matrix. for all , namely, , and also sets the block size , see, e.g., [61, 62] for more discussion on block coherence and its related theoretical investigation. Equipped with the block coherence, we here consider extending Theorem 1 to guarantee the robust recovery of such structured block-sparse signals corrupted with the bounded noise by solving the following unconstrained problem
(31) |
This problem sometimes is called the group Lasso [63], and it can also be viewed as the block (group) extension of (5). One can find our second extension of Theorem 1 as follows.
Comments
There are no comments yet.