1 Introduction
Recent emerging applications such as visual sensor surveillance and mobile augmented reality are challenging in a distributed sensing scenario of smart tiny devices within heterogeneous networks in realtime. By cooperatively using multiple sensors, we can further enhance gathering and processing information under timeresource constraints for such devices. In particular, multiview object tracking and recognition [1, 2] are challenging how to utilize correlations among the multiview images without communications among them. The distributed sensing challenge may be addressed by representing it as a distributed sparse representation of multiple sources [3]. This raises the needs for representing and reconstructing the sparse sources along with exploiting the correlated information among them. One of the key problems is how to robustly reconstruct a compressed source given supported observations, i.e., other available sources as SI. Attractive CS techniques [4, 5, 6, 7, 8, 9, 10, 11, 12] may enable these distributed sources to deal with such reconstruction problems.
Recently, some CS reconstructions with prior information [7, 8, 9, 10, 11, 12] have been investigated in terms of theoretical as well as application aspects. It was shown that if the prior information is good enough, a  minimization [7, 8], which integrates SI into classical CS, improves the reconstruction dramatically. The work in [9, 10, 11, 12] shows efficient reconstructions with SI for sparse signals from a limited number of measurements in background subtraction and MRI imaging applications. These schemes are restricted to considering only one SI in conditions of good enough qualities. However, we are interested in the multiview heterogeneous sources with quality changing in time, i.e., unknown prior correlations, correspondences, and SI qualities among sources. Therefore, we are aiming at improving the reconstruction given arbitrary SIs, whose possible poor qualities and the multiple SIs are taken into account.
In this work, we propose an efficient reconstruction algorithm with multiple SIs using adaptive weights by solving a weighted  problem. The algorithm adaptively reconstructs a sparse source from a small number of measurements, which are obtained by a random projection, given other sources. Our previous work [13] proposed a reconstruction algorithm with multiple SI, where the weights are iteratively updated for each given individual index cross all SIs under only its local constraint. To improve performance, the algorithm we propose here contributes to solving the  problem that computes the adaptive weights in two levels. IntraSI weights are first computed in each individual SI then global interSI weights are updated at every iteration of the reconstruction. By this way, the proposed algorithm can robustly take advantage of exploiting both intrasource and intersource correlations to adapt to onthefly correlation changes in time, e.g., multiview images with different SI qualities.
The rest of this paper is organized as follows. Section 2 reviews related works including fundamental problems of signal recovery and CS with SI. Our problem statement and proposed algorithm are presented in Sec. 3. We demonstrate performances of the proposed algorithm on distributed sparse sources in Sec. 4.
2 Related Work
In this section, we review a fundamental problem of signal recovery from lowdimensional signals [4, 5, 6] and CS with SI [7, 8, 9, 10, 11, 12].
Signal Recovery. Lowdimensional signal recovery arises in a wide range of applications in signal processing. Most signals in such applications have sparse representations in some domain. Let denote a sparse source, which is indeed compressible. The source can be reduced by sampling via a projection [5]. We denote a random measurement matrix for by
, whose elements are sampled from an i.i.d. Gaussian distribution. Thus, we get a compressed vector
, also called measurement, consisting of elements. The source can be recovered [5, 4] by solving:(1) 
where is norm of wherein is an element of .
Problem (1) becomes finding a solution to
(2) 
where is a smooth convex function with Lipschitz constant [6] of gradient and is a continuous convex function possibly nonsmooth. Problem (1) is obviously a special case of (2) with , where is a regularization parameter, and . The results of using proximal gradient methods [6] give that at iteration can be iteratively computed by:
(3) 
where and is a proximal operator that is defined by
(4) 
CS with SI. CS with prior information or SI via  minimization is improved dramatically if SI has good enough quality [7, 8, 10]. The  minimization considers reconstructing given a SI, by solving the problem (2) with , i.e., solving:
(5) 
The  minimization (5) has an expression for the bound [7, 8, 10] on the number of measurements required to successfully reconstruct that is a function [7, 8, 10] of the quality of SI .
3 Reconstruction with Multiple SI using Adaptive Weights
3.1 Problem Statement
We consider a problem statement regarding how to efficiently reconstruct a sparse source given multiple SIs. The reconstruction for the distributed sensing of sparse sources is motivated by an idea of how multiple heterogeneousdynamic lightweight cameras can perform robustly on the desired applications under resourcetime constraints, where sensing and processing are expensive. CS [4, 5, 6] is emerged as an elegant technique to deal with these challenges for such realtime sensing and processing. Despite prohibiting the communication between tiny cameras, we can take advantage of their correlations at the decoder through supported SIs. The known SIs can be some other reconstructed sources existing at the decoder, where we can exploit intersource redundancies for a robust reconstruction from the reduced sparse source. Therefore, we need an advanced reconstruction flexibly adapting to onthefly changes according to the heterogeneous sparse sources.
Let us consider the example scenario of the distributed compressed sensing of tiny cameras for multiview object recognition in Fig. 1, where a corresponding feature histogram acquired from a given camera is considered as a sparse source (Fig. 1(b)). Figure 1 shows threeview images, View 1 (Fig. 1(a)), View 2 (Fig. 1(c)), View 3 (Fig. 1(e)), of the given object 60 in the COIL100 database [14] with corresponding SIFTfeature [15] points to create feature histograms as sparse sources. The feature histograms are created by extracting all SIFT [15] features from the image then propagating down a hierarchical vocabulary tree based on a hierarchical means algorithm [16]. In reality, we may need very highdimensional histograms, it is essential to reduce the source dimension by CS before further processing. The idea of CS is reducing the source without prior knowledge of the source distribution. Thus a reduced , which is obtained by compressing 1000 dimensions (D) (Fig. 1(b)), is to be conveyed to the joint decoder. We use the available compressed to reconstruct with given known SIs, 1000D (Fig. 1(d))and 1000D (Fig. 1(f)), which are feature histograms of neighbor views. Obviously, are naturally correlated in some degree.
In order to reconstruct from given , we may go straight to the solution (5)  minimization with only one input SI either or . The measurements of required to successfully reconstruct is a function of the quality of . There may be a chance that the  reconstruction performs worse than the reconstruction, incurred by a not good enough SI, e.g., , exposing the drawback of the  reconstruction. Therefore, we come up with a new Reconstruction Algorithm with Multiple Side Information using Adaptive weights (RAMSIA) uses the measurement , which is obtained by , and known SIs, as inputs. The final goal is to reconstruct output as best as we can.
RAMSIA is built up using the advances of the  minimization (5) and additionally adaptive weights to robustly work on multiple SIs. The objective function of RAMSIA shall be created as an  minimization based on the problem in (2) with
(6) 
where are weights among SIs and are a diagonal weight matrix for each SI, , wherein is the weight in at index for a given SI . To conform the first term to the remaining terms, we rewrite it as where , thus from (6). We may emphasize as a difference compared with our previous work [13] that we here compute weights in two levels. First for intraSI weights and then interSI weights among SIs as global weights are computed to optimize the weights. We end up formulating the objective function of the  problem by:
(7) 
It can be noted that this (7) would become the problem (5) if , , and , where is a unit diagonal matrix (size of ), .
3.2 The proposed RAMSIA
Before solving the  problem in (7), we need to address an arisen question how to determine the weight values to improve the reconstruction as well as avoid declination by the negative SIs to the reconstruction. We should distribute relevant weights not only in the intraSI but also inter SIs. To optimize the objective function in (7), we impose constraints on both all and , by this way, we will be able to assign weights on multiple SIs according to their qualities during the iterative process. We propose to solve the  problem in (7) based on the proximal gradient method [6], i.e., at every iteration we need to update, on the one hand, the weights and on the other hand compute . We may have different strategies to update the weights depending on our constraints.
To solve the minimization problem in (7), we propose SI constraints by at the intra SI level and at the inter SI level. We minimize the objective function in (7) in two steps, one for and the remain for during the iterative reconstruction. Referring to our previous work [13], a different weighting strategy was used, where at a given index cross all SIs and all .
Firstly, by considering and fixed, for each SI we compute by optimizing (7), i.e., is equal to
(8) 
where is an element of at index . We can achieve the minimization of (8) (following Cauchy inequality) when all are equal to a positive parameter , i.e., with a small added . Combining with the intra SI constraint , we get intra SI weights by:
(9) 
Secondly, given and , we compute at the interSI level by optimizing the problem in (7):
(10) 
Similar to (8), from (10) we obtain as this formula:
(11) 
Combining (11) with the inter SI constraint , we get the inter SI weights by:
(12) 
Given and , we compute at iteration to minimize the problem in (7). Based on the proximal gradient method [6], is obtained from (3), where . The remaining key question is how to compute the proximal operator . From (26) (derived in Appendix), is obtained by:
(13) 
Finally, we sum up the proposed RAMSIA in Algorithm 1 based on a fast iterative FISTA algorithm [6], where the main difference from FISTA is the iterative computation of the updated weights. RAMSIA iteratively updates two weight levels in turn in (9) and in (12). It can be noted that the Stopping criteria in Algorithm 1 can be either a maximum iteration number , a relative variation of the objective function (7
), or a change of the number of nonzero components of the estimate
. In this work, the relative variation of (7) is chosen.4 Experimental Results
Success probabilities of reconstructing the original 1000D
vs. number of dimensions of the compressed using 1,2,3 SIs.First we consider the reconstruction of multiview feature histograms as sparse sources , which could be used in multiview object recognition. Given an image, its feature histogram is formed as in Sec. 3.1, e.g., . The size of the vocabulary tree [16] depends on the value of and number of hierarchies, e.g., if and 3 hierarchies, vocabularies as 1000D, which may be a very big number in reality. Because of the small number of features in a single image, the histogram vector is sparse. Therefore, is first projected into the compressed vector that is to be sent to the decoder. At the joint decoder, we suppose is to be reconstructed given some already decoded histograms of neighbor views, e.g., ,,. In this work, we use the COIL100 [14] containing multiview images of 100 small objects with different angle degrees. For the sake of ensuring our experimental setup a realistic circumstance of multiple distributed sources without given correspondences, we randomly select the 4 neighbor views (3 neighbors as SIs) of representative objects 16 (Fig. 2(a)) and 60 (Fig. 2(b)) over 72 views captured through 360 degrees in the COIL100 [14] multiview database. In addition, we consider the reconstruction of a generated sparse source given three known SIs (Fig. 2(c)). We generate with with 100 supports (number of nonzeros), which is generated from the standard i.i.d. Gaussian distribution. To illustrate a scenario of the less correlated SIs, we generate SIs with all supports of subtractions, , , are equal to 300.
We will evaluate and compare the reconstruction accuracy of the proposed RAMSIA exploiting different number of SIs against the existing reconstructions. Let RAMSIA denote the RAMSIA reconstruction, where indicates number of SIs, e.g., RAMSIA, RAMSIA, RAMSIA are RAMSIA using 1 (), 2 (,), and 3 (,,) SIs, respectively. Similarly, RAMSI, RAMSI, RAMSI are three versions in our previous work [13]. Let FISTA denote the  CS reconstruction with one SI () [7]. The existing FISTA [6] and FISTA reconstructions [7] are used for comparison. The original source 1000D is compressed into different lowerdimensions . We assess the accuracy of a reconstructed versus the via the success probability, denoted as , versus the number of dimensions. For a fixed dimension, the is the number of times, in which the source is recovered as with an error , divided by the total 100 trials (each trial considered randomly selecting 4 neighbor views for objects 16,60 (Figs. 2(a)+2(b)) and different generated ,, for generated signals (Fig. 2(c))). RAMSIA (Algorithm 1) thus has , , and we set parameters , .
Figure 2 presents the performances of the proposed RAMSIA as well as RAMSI [13] with 1,2,3 SIs, FISTA [6], and FISTA [7] in terms of the success probabilities versus dimensions. It shows clearly that RAMSIA significantly improves the reconstruction accuracy. Furthermore, the results using 3 SIs, RAMSIA3 gives highest accuracy and the performance of RAMSIA2 is higher than that of RAMSIA1. For instance, RAMSIA requires 300, 320, 340 measurements corresponding to using 3, 2, 1 SIs to successfully reconstruct Object 16 (Fig. 2(a)). Despite exploiting only the same one SI, RAMSIA1 is absolutely better than FISTA. For the signal in Fig. 2(c), the accuracies of FISTA are worse than those of FISTA, i.e., SI does not help, however, RAMSIA still significantly outperforms FISTA. These results may highlight the drawback of  minimization when SI qualities are not good enough. Therefore, if we exploit the weighted  minimization as RAMSIA, our reconstruction can deal with the negative instances by adaptive weights, e.g., less weights on such negative cases of SI , to achieve the improvements for all scenarios.
5 Conclusion
This paper proposed an sparse signal reconstruction with multiple SIs for the distributed sources by solving the general  minimization problem. The proposed RAMSIA scheme iteratively updated the adaptive weights in two levels not only in each intraSI but among SIs to adaptively optimize the reconstruction. RAMSIA took advantage of exploiting both intrasource and intersource redundancies to adapt to the changes of the heterogeneous sparse sources to deal with the different SI qualities. We experimentally tested RAMSIA on the multiple feature histograms as multiview sparse sources captured from distributed cameras and also the generated signals. The results showed that RAMSIA robustly outperformed the conventional CS and recent  minimization methods. Moreover, RAMSIA with higher number of SIs gained more improvements than the one with smaller number of SIs.
6 Appendix
We shall compute the proximal operator (4) with . From (4), is expressed by:
(14) 
We may note that both terms in (14) are separable in and thus we can minimize each element of individually as
(15) 
We consider the . Without loss of generality, we assume . For convenience, let us denote and . When is located in one of the intervals, we suppose with , where exists. Taking the derivative of in delivers
(16) 
where is a sign function. In addition, let denote a boolean function, i.e., if , otherwise . Consequently, and from (16), we rewrite:
(17) 
When setting to minimize , we derive:
(18) 
This (18) is only valid in , i.e.,
(19) 
In case of that does not belong to alike intervals in (19), i.e.,
(20) 
We will prove that (15) is minimum when in the following Lemma 6.1.
Proof.
We reexpress (15) by:
(21) 
Applying a simple inequality , where , to the first term and expanding the second term in (21), we obtain:
(22) 
It can be noted that . Thus the inequality in (22) is equivalent to:
(23) 
Without difficulty, from the expression in (20), we get:
(24) 
Eventually, we observe that the part including in the right hand side of the inequality in (23) is
(25) 
With (24), the expression in (25) is minimum when . Therefore, we deduce that (15) is minimum when . ∎
References
 [1] M. Taj and A. Cavallaro, “Distributed and decentralized multicamera tracking,” IEEE Signal Process. Mag., vol. 28, no. 3, pp. 46–58, 2011.
 [2] A. Sankaranarayanan, A. Veeraraghavan, and R. Chellappa, “Object detection, tracking and recognition for multiple smart cameras,” Proc. of IEEE, vol. 96, no. 10, pp. 1606–1624, 2008.
 [3] A. Y. Yang, M. Gastpar, R. Bajcsy, and S. Sastry, “Distributed sensor perception via sparse representation,” Proc. of IEEE, vol. 98, no. 6, pp. 1077–1088, 2010.
 [4] D. Donoho, “For most large underdetermined systems of linear equations the minimal norm solution is also the sparsest solution,” Communications on Pure and Applied Math, vol. 59, no. 6, pp. 797–829, 2006.
 [5] E. Candès and T. Tao, “Nearoptimal signal recovery from random projections: Universal encoding strategies?” IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5406–5425, Apr. 2006.
 [6] A. Beck and M. Teboulle, “A fast iterative shrinkagethresholding algorithm for linear inverse problems,” SIAM Journal on Imaging Sciences, vol. 2(1), pp. 183–202, 2009.
 [7] J. F. Mota, N. Deligiannis, and M. R. Rodrigues, “Compressed sensing with side information: Geometrical interpretation and performance bounds,” in IEEE Global Conf. on Signal and Information Processing, Austin, Texas, USA, Dec. 2014.
 [8] ——, “Compressed sensing with prior information: Optimal strategies, geometry, and bounds,” ArXiv eprints, Aug. 2014.
 [9] J. Scarlett, J. Evans, and S. Dey, “Compressed sensing with prior information: Informationtheoretic limits and practical decoders,” IEEE Trans. Signal Process., vol. 61, no. 2, pp. 427–439, Jan. 2013.
 [10] J. F. Mota, N. Deligiannis, A. Sankaranarayanan, V. Cevher, and M. R. Rodrigues, “Dynamic sparse state estimation using  minimization: Adaptiverate measurement bounds, algorithms and applications,” in IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Brisbane, Australia, Apr. 2015.
 [11] L. Weizman, Y. C. Eldar, and D. B. Bashat, “Compressed sensing for longitunal MRI: An adaptiveweighted approach,” Medical Physics, vol. 42, no. 9, pp. 5195–5207, 2015.
 [12] G. Warnell, S. Bhattacharya, R. Chellappa, and T. Basar, “Adaptiverate compressive sensing using side information,” IEEE Trans. Image Process., vol. 24, no. 11, pp. 3846–3857, 2015.
 [13] H. V. Luong, J. Seiler, A. Kaup, and S. Forchhammer, “A reconstruction algorithm with multiple side information for distributed compression of sparse sources,” in Data Compression Conference, Snowbird, Utah, USA, Apr. 2016.
 [14] S. A. Nene, S. K. Nayar, and H. Murase, “Columbia object image library (coil100),” Technical Report CUCS00696, Feb. 1996.

[15]
D. G. Lowe, “Object recognition from local scaleinvariant features,” in
IEEE Int. Conf. on Computer Vision
, Kerkyra, Greece, Sep. 1999. 
[16]
D. Nistér and H. Stewénius, “Scalable recognition with a vocabulary
tree,” in
IEEE Int. Conf. on Computer Vision and Pattern Recognition
, New York, USA, Jun. 2006.
Comments
There are no comments yet.