Compressive Shift Retrieval

03/20/2013
by   Henrik Ohlsson, et al.
0

The classical shift retrieval problem considers two signals in vector form that are related by a shift. The problem is of great importance in many applications and is typically solved by maximizing the cross-correlation between the two signals. Inspired by compressive sensing, in this paper, we seek to estimate the shift directly from compressed signals. We show that under certain conditions, the shift can be recovered using fewer samples and less computation compared to the classical setup. Of particular interest is shift estimation from Fourier coefficients. We show that under rather mild conditions only one Fourier coefficient suffices to recover the true shift.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

03/25/2015

Compressed sensing MRI using masked DCT and DFT measurements

This paper presents modification of the TwIST algorithm for Compressive ...
12/03/2018

On learning with shift-invariant structures

We describe new results and algorithms for two different, but related, p...
08/06/2021

Shift-invariant waveform learning on epileptic ECoG

Seizure detection algorithms must discriminate abnormal neuronal activit...
01/02/2020

Identifiability Conditions for Compressive Multichannel Blind Deconvolution

In applications such as multi-receiver radars and ultrasound array syste...
01/29/2013

Quadratic Basis Pursuit

In many compressive sensing problems today, the relationship between the...
07/06/2016

Accelerating eigenvector and pseudospectra computation using blocked multi-shift triangular solves

Multi-shift triangular solves are basic linear algebra calculations with...
05/21/2014

Compressive Sampling Using EM Algorithm

Conventional approaches of sampling signals follow the celebrated theore...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Shift retrieval is a fundamental problem in many signal processing applications. For example, to map the ocean floor, an active sonar can be used. The sonar transmits certain sound pulse patterns in the water, and the time it takes to receive the echoes of the pulses indicates the depth of the ocean floor. In target tracking using two acoustic sensors, the time shift when a sound wave of a vehicle reaches the microphones indicates the direction to the vehicle. In the case of a time shift, the shift retrieval problem is often referred to as time delay estimation (TDE) [3]

. In computer vision, the spatial shift relating two images is often sought and referred to as image registration or alignment

[4, 5, 6].

Traditionally, the shift retrieval problem is solved by maximizing the cross-correlation between the two signals [7]. In this paper, we revisit this classical problem, and show how the basic premise of compressive sensing (CS) [8, 9, 10, 11] can be used in the context of shift retrieval. This allows to recover the shift from compressed data leading to computational and storage savings.

Compressive sensing is a sampling scheme that makes it possible to sample at the information rate instead of the classical Nyquist rate predicted by the bandwidth of the signal [12]. The majority of the results in compressive sensing discuss conditions and methods for guaranteed reconstruction from an under-sampled version of the signal. Therefore, the information rate is typically referred to as the one that guarantees the recovery of the sparse signal.

However, for many applications such as the aforementioned examples in shift retrieval, obtaining the signal may not be needed. The goal is to recover some properties or statistics of the unknown signal. Taking the active sonar for example, one may wonder if it is really necessary to sample at a rate which is twice that of the bandwidth of the transmitted signal so that the received signal can be exactly reconstructed? Clearly the answer is no. Since the signal itself is not of interest to the application, we might consider an alternative sampling scheme to directly estimate the shift without first reconstructing the signal. These ideas have in fact been recently explored in the context of radar and ultrasound [13, 14, 15, 16] with continuous time signals and multiple shifts. Here we consider a related problem and ask: What is the minimal information rate to shift retrieval when two related discrete-time signals are under-sampled?

It turns out that under rather mild conditions, we only need fractions of the signals. In fact, we will show that only one Fourier coefficient from each of the signals suffices to recover the true shift. We refer to the method as compressive shift retrieval (CSR). It should be made clear that CSR does not assume that any of the involved signals are necessarily sparse.

As the main contribution of the paper, we will show that when the sensing matrix is taken to be a partial Fourier matrix, under suitable conditions, the true shift can be recovered from both noise-free and noisy measurements using CSR. Furthermore, CSR reduces both the computational load and the number of samples needed in the process. This is of particular interest since recent developments in sampling [17, 18, 19] have shown that Fourier coefficients can be efficiently obtained from space (or time) measurements by the use of an appropriate filter and by subsampling the output. Remarkably, our results also show that in some cases sampling as few as one Fourier coefficient is enough to perfectly recover the true shift.

I-a Prior Work

Compressive signal alignment problems have been addressed in only a few publications and, to the authors’ best knowledge, not in the same setup studied in this paper. In [20], the authors considered alignment of images under random projection. The work was based on the Johnson-Lindenstrauss property of random projection and proposed an objective function that can be solved efficiently using difference-of-two-convex programming algorithms. In this paper, we instead focus on proving theoretical guarantees of exact shift recovery when the signal is subsampled by a partial Fourier basis. The theory developed in [20] does not apply to this setup. The smashed filter [21]

is another related technique. It is a general framework for maximum likelihood hypothesis testing and can be seen as a dimensionally reduced matched filter. It can therefore be applied to the shift retrieval problem. The underlying idea of both the smashed filter and CSR are the same in that both approaches try to avoid reconstructing the signal and extract the sought descriptor, namely, the shift, from compressive measurements. However, the analysis and assumptions are very different. For CSR, we develop requirements for guaranteed recovery of the true shift for a given measurement matrix. For the smashed filter, the analysis focuses on random orthoprojections and provides probabilities for correct recovery as a function of the number of projections. Also, in this paper, we are particularly interested in Fourier measurements, and many of the results are therefore tailored to this setting. The work presented here can therefore be seen as complementary to what was presented in

[21] and its extension in [22].

I-B Notation and Assumptions

We use normal fonts to represent scalars and bold fonts for vectors and matrices. The notation represents the absolute value for scalars, vectors and matrices, and it returns the cardinality of a set if the argument is a set. For both vectors and matrices, is the -norm function that returns the number of nonzero elements of its argument. Similarly, represents the -norm. For a vector , the -norm is defined as , where is the th element of . For a matrix , is defined as , where is the -th element of . Furthermore, denotes the complex conjugate transpose of . Let denote an identity matrix, an matrix of zeros, and be the set of integers. returns the real part of its argument. We say that two -dimensional vectors and are related by an cyclic-shift if , where is defined as

(1)

Throughout the paper, we will assume that the shift is unique up to a multiple of . Also note that we are considering cyclic shifts.

I-C Organization

In Sections II and III, we study the CSR problem under the assumption that the measurements are noise free. Next, we extend the results to noisy measurements in Section IV. As we are particularly interested in Fourier measurements, we will tailor the results to this particular choice of sensing matrix. Section V concludes the paper. All proofs are provided in the Appendice for clarity.

Ii Noise-Free Compressive Shift Retrieval

The shift retrieval problem is a multi-hypothesis testing problem: Define the th hypothesis , , as

and accept if and otherwise reject. Since the true shift is assumed unique, only one hypothesis will be accepted and the corresponding shift is necessarily the true solution. The connection to the cross-correlation is now trivial:

(2)

where we use the fact that . Since , equating is equivalent to minimizing or maximizing the real part of the cross-correlation with respect to :

(3)

Now, assume that the compressed measurement signals and are given and related to the ground-truth signals and its shifted version via the sensing matrix , . The goal of CSR is to recover the shift relating and from the compressed measurements and .

Since only the compressed measurements and are assumed available, we can not evaluate or maximize for each hypothesis . However, if and commute for all , then

(4)

Hence, we could consider the test:

(5)

It is clear that if is such that , then will also hold. However, the other way around might not be true. Therefore, we might erroneously accept a wrong hypotheses using (5). The next theorem lists the conditions by which the testing (5) is guaranteed to accept the correct hypothesis. Notice that testing the condition is equivalent to minimizing with respect to .

Theorem 1 (Shift Recovery from Low-Rate Data).

Let be an matrix with the th column equal to , and define . If the sensing matrix satisfies the following conditions:

  • ,

  • and

  • all columns of are different,

then

(6)

or equivalently the test (5) recovers the true shift.

The conditions of Theorem 1 may seem restrictive. However, as we will show in Lemma 3, if is chosen as a partial Fourier matrix, then the first two conditions of Theorem 1 are trivially satisfied. The last condition is the only one that needs to be checked and will lead to a condition on the sampled Fourier coefficients.

The conditions of Theorem 1 can be checked prior to estimating the shift. However, knowing the estimate of the shift, it is easy to see from the proof (see the proof of Lemma 8) that it is enough to check if the column of associated with the estimate of the shift is different than all the other columns of . Hence, we do not need to check if all columns of are different. This conclusion is formulated in the following corollary, which is less conservative than Theorem 1.

Corollary 2 (Test for True Shift).

Let be an matrix with the th column equal to , and define . If the sensing matrix satisfies the following conditions:

  • , and

  • ,

then

(7)

is the true shift if the th column of is different than all the other columns of .

Iii Compressive Shift Retrieval using Fourier Coefficients

Of particular interest is the case where is made up of a partial Fourier basis. That is, takes the form

where . For this specific choice,

where denotes the

th Fourier coefficient of the Fourier transform of

.

For a sensing matrix made up by a partial Fourier basis, we have the following useful result:

Lemma 3.

Let be a partial Fourier matrix. Then for all .

Using this result in Theorem 1 gives the following corollary:

Corollary 4 (Shift Recovery from Low Rate Fourier Data).

With denoting a partial Fourier matrix and and the th element of and ,

(8)

recovers the true shift if there exists such that and contains no integers. In particular, measuring only the first Fourier coefficients () of and would, as long as the coefficients are nonzero, suffice to recover the true shift.

Remarkably, in the extreme case when , the corollary states that all we need is two scalar measurements, and , to perfectly recover the true shift. The scalar measurements can be any nonzero Fourier coefficient of and as long as contains no integers. As noted in the corollary, the first Fourier coefficients () of and would suffice. Also note that only multiplications are required to evaluate the test. This should be compared with multiplications to evaluate the cross-correlation for the full uncompressed signals and . Corollary 4 is easy to check but more conservative than both Theorem 1 and Corollary 2.

To validate the results, we carried out the following example. In each trial we let the sample dimension and the shift be random integers between 1 and 9 and generate by sampling from a

-dimensional uniform distribution. We let

and make sure that in each trial is a partial Fourier basis satisfying the assumptions of Corollary 4. We carry out 10000 trials. The true shift is successfully recovered in each trial by the simplified test (8), namely, with 100% success rate. This is quite remarkable since when , we recover the true shift using only two scalar measurement and and of the multiplications that maximizing the real part of inner product between the original signals (3) would need.

Iv Noisy Compressive Shift Retrieval

Now we consider the noisy version of compressive shift retrieval, where the measurements and are perturbed by noise:

(9)

Similar to the noise-free case, here we can also guarantee the recovery of the true shift. Our main result is given in the following theorem:

Theorem 5 (Noisy Shift Recovery from Low-Rate Data).

Let be such that , the th column of be shifted versions of , and assume that is a partial Fourier matrix and that the noisy measurements are used in (8) to estimate the shift. If the -norm difference between any two columns of is greater than

then the estimate of the shift is not affected by the noise.

Note that the theorem only states that the noise does not affect the estimate of the shift. It does not state that the shift will be the true shift.

We illustrate the results by running a Monte Carlo simulation consisting of 10000 trials for each sample dimension and for two different SNR levels. In Figure 1, 10 histograms are shown (corresponding to ) for the

(10)

being 2 (low SNR) and in Figure 2, (high SNR). The errors and were both generated by sampling from

(11)

We further use , and sample from a uniform (0,1)-distribution. The conclusion from the simulation is that the smaller the , the more the estimate of the shift is sensitive to noise. Notice that when the test (8) reduces to the classical test of maximizing the cross-correlation.

We can now use Theorem 5 to check if the noise affected the estimate of the shift or not in each of the trials. For and the high SNR, 40% of the trials satisfied the conditions of Theorem 5 and the noise therefore did not affect the shift estimates in those cases. Of the trails that satisfied the conditions, all predicted the true shift and none a false shift. Note however that Theorem 5 only states that if the conditions are satisfied, then the estimated shift is the same as if we would have used the noise free compressed measurements in the test (8). It does not state that the estimate will be the true shift.

Fig. 1: Histogram plots for the estimated shift and low SNR. From left to right, top to bottom, . The true shift was set to 5 in all trials.
Fig. 2: Histogram plots for the estimated shift and high SNR. From left to right, top to bottom, . The true shift was set to 5 in all trials.

Theorem 5 gives conditions for when the noise does not affect the estimate of the shift. This is a good property but even better would be if the recovery of the true shift could be guaranteed. This is given by the following corollary.

Corollary 6 (Recovery of the True Shift from Noisy Low-Rate Data).

If the -norm difference between any two columns of is greater than and the conditions of Theorem 5 are fulfilled, then (8) recovers the true shift.

If the estimate of the shift has been computed, a less conservative test can be used to check if the computed estimate has been affected by noise and if it is the true one. We summarize our conclusion in the following corollary.

Corollary 7 (Test for True Shift in the Presence of Noise).

Assume that (8) gives as an estimate of the shift. If the difference between any column and the -column of is greater than and , then is the true shift.

V Conclusion

To recover the cyclic shift relating two 1D signals, the cross-correlation is usually evaluated for all possible shifts. Recent advances in hardware, signal acquisition and signal processing have made it possible to sample or compute Fourier coefficients of a signal efficiently. It is therefore of particular interest to see under what conditions the true shift can be recovered from the Fourier coefficients. We have proposed a criterion that is computationally more efficient than using the time samples, and we have shown that the true shift can be recovered using as few as one Fourier coefficient. We have also derived bounds for perfect recovery for both noise free and noisy measurements.

Appendix A Proofs: Noise-Free Compressive Shift Retrieval

Before proving the Theorem 1, we state two lemmas.

Lemma 8 (Recovery of Shift using Projections).

Let be the -matrix made up of cyclically shifted versions of as columns. If the columns of are distinct, then the true shift can be recovered by

(12)
Proof of Lemma 8.

Since the shift relating and is assumed unique, it is clear that the true shift is recovered by

(13)

Assume that the solution of (12) is not equivalent to that of (13). Namely, assume that (13) gives , (12) gives and . Since will give a zero objective value in (12), so must . We therefore have that and hence

(14)

Since , , and , implies that two columns of are identical. This is a contradiction and we therefore conclude that both (12) and (13) recover the true shift. ∎

Lemma 9 (From (12) to (6)).

Under conditions 1) and 2) of Theorem 1, the shifts recovered by (12) and (6) are the same.

Proof of Lemma 9.

Consider the objective of (12):

(15)

Writing , problem (12) is equal to

(16)

Now, if and using that for a shift matrix, then

(17)

which is independent of . Therefore, the shift recovered by (16) is the same as that of

(18)

Lastly, if we again use that and , then (6) follows from

(19)
(20)
(21)
(22)

where and . ∎

We are now ready to prove Theorem 1.

Proof of Theorem 1.

The assumptions of Theorem 1 imply that requirements of both Lemmas 8 and 9 are satisfied. The theorem therefore follows trivially. ∎

We next prove Corollary 2.

Proof of Corollary 2.

In the proof of Lemma 8, leads to if the columns of were all distinct. Now, if

(23)

that corresponds to the th element of being one and all other elements zero. Hence, Lemma 8 can be made less conservative if is known by requiring that only the th column of is different than all other columns. ∎

Proof of Lemma 3.

Let and . By the definition of , is a column permutation of where the columns are shifted times to the right. Thus, the th column of is equal to the th column of where . It is also easy to see that permutes the columns of by to the left so that the th column of is equal to the th column of where . Now, the th element of is given by

(24)

where is used to denote the th column of and the th column or . On the other hand, the -th element of is given by

(25)

Clearly, the two are equivalent. ∎

We are now ready to prove Corollary 4.

Proof of Corollary 4.

Lemma 3 gives that Condition 1) of Theorem 1 is satisfied. Since a full Fourier matrix is orthonormal, a matrix made up of a selection of rows of a Fourier matrix satisfies Condition 2). The last condition of Theorem 1 requires columns of to be distinct. A sufficient condition is that there exists a row with all distinct elements. As shown previously, the -th element of is If is assumed nonzero, a sufficient condition for to have distinct columns is that . This condition can be simplified to . By realizing that takes values in we get that the condition is equivalent to requiring that there is no integers in . Due to symmetry, a sufficient condition for distinct columns is that there exists a such that and contains no integers. Lastly, if we write out we get that the th element is equal to and hence the simplified test proposed in (8). ∎

Appendix B Proofs: Noisy Compressive Shift Retrieval

Proof of Theorem 5.

From Lemma 9 we can see that seeking that maximizes is equivalent to seeking that solves

(26)

where the first column of is equal to (which defines the first column of ) and the th column of a circular shift of the first column steps. Assume that solves (26). Since our measurements are noisy, we can not expect a zero loss. The loss can be shown given by

(27)

Now, consider . Assume that solves the noise-free version of (26) and let . We have the following inequality:

where we used the fact that . Therefore

(28)

Since , if the difference between any two columns of is greater than , then . ∎

Proof of Corollary 6.

Let and be any vectors such that , and . Using the triangle inequality we have that

(29)
(30)
(31)

Hence, if , then is greater than zero. Now since Theorem 5 gives that (8) recovers the same shift as if the measurements would have been noise-free, and since Theorem 1 gives that the noise-free estimate is equal to the true shift if is greater than zero (or equivalent that all columns of are distinct), we can guarantee the recovery of the true shift also in the noisy case. ∎

Proof of Corollary 7.

The corollary follows trivially by setting the th element of to one and all other elements zero in the proofs of Theorem 5 and Corollary 6. ∎

References

  • [1] H. Ohlsson, Y. C. Eldar, A. Y. Yang, and S. S. Sastry, “Compressive shift retrieval,” in the 38th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vancouver, Canada, May 2013.
  • [2] ——, “Compressive shift retrieval,” CoRR, vol. abs/1303.4996, 2013.
  • [3] G. Carter and C. Knapp, “Time delay estimation,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 1976.
  • [4] B. Lucas and T. Kanade, “An iterative image registration technique with an application in stereo vision,” in

    Int’l J. Conf. Artificial Intelligence

    , 1981.
  • [5]

    G. Hager and P. Belhumeur, “Efficient region tracking with parametric models of geometry and illumination,”

    IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10, pp. 1025–1039, 1998.
  • [6] Y. Peng, A. Ganesh, J. Wright, and Y. Ma, “RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images,” in

    Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition

    , 2010.
  • [7] A. Hero, H. Messer, J. Goldberg, D. Thomson, M. Amin, G. Giannakis, A. Swami, J. Tugnait, A. Nehorai, A. Swindlehurst, J.-F. Cardoso, L. Tong, and J. Krolik, “Highlights of statistical signal and array processing,” IEEE Signal Processing Magazine, vol. 15, no. 5, pp. 21–64, 1998.
  • [8]

    E. J. Candès and T. Tao, “Decoding by linear programming,”

    IEEE Transactions on Information Theory, vol. 51, no. 12, 2005.
  • [9] E. J. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Transactions on Information Theory, vol. 52, pp. 489–509, Feb. 2006.
  • [10] D. Donoho, “Compressed sensing,” IEEE Transactions on Information Theory, vol. 52, no. 4, pp. 1289–1306, Apr. 2006.
  • [11] Y. C. Eldar and G. Kutyniok, Compressed Sensing: Theory and Applications.   Cambridge University Press, 2012.
  • [12] M. Mishali and Y. C. Eldar, “Sub-Nyquist sampling,” IEEE Signal Processing Magazine, vol. 28, no. 6, pp. 98–124, 2011.
  • [13] T. Chernyakova and Y. C. Eldar, “Fourier domain beamforming: The path to compressed ultrasound imaging,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, Jul. 2013, submitted.
  • [14] O. Bar-Ilan and Y. C. Eldar, “Sub-Nyquist radar via doppler focusing,” IEEE Transactions on Signal Processing, Nov. 2012, submitted.
  • [15] E. Baransky, G. Itzhak, I. Shmuel, N. Wagner, E. Shoshan, and Y. C. Eldar, “A sub-Nyquist radar prototype: Hardware and algorithms,” IEEE Transactions on Aerospace and Electronic Systems, special issue on Compressed Sensing for Radar, Aug. 2012, submitted.
  • [16] M. Vetterli, P. Marziliano, and T. Blu, “Sampling signals with finite rate of innovation,” IEEE Transactions on Signal Processing, vol. 50, no. 6, pp. 1417–1428, Jun. 2002.
  • [17] R. Tur, Y. C. Eldar, and Z. Friedman, “Innovation rate sampling of pulse streams with application to ultrasound imaging,” IEEE Transactions on Signal Processing, vol. 59, no. 4, pp. 1827–1842, Apr. 2011.
  • [18] K. Gedalyahu, R. Tur, and Y. C. Eldar, “Multichannel sampling of pulse streams at the rate of innovation,” IEEE Transactions on Signal Processing, vol. 59, no. 4, pp. 1491–1504, Apr. 2011.
  • [19] E. Baransky, G. Itzhak, I. Shmuel, N. Wagner, E. Shoshan, and Y. C. Eldar, “A sub-Nyquist radar prototype: Hardware and algorithms,” IEEE Transactions on Aerospace and Electronic Systems, special issue on Compressed Sensing for Radar, Aug. 2012, submitted to.
  • [20] E. Kokiopoulou, D. Kressner, and P. Frossard, “Optimal image alignment with random measurements,” in EUSIPCO, 2009.
  • [21] M. A. Davenport, M. F. Duarte, M. Wakin, J. N. Laska, D. Takhar, K. Kelly, and R. G. Baraniuk, “The smashed filter for compressive classification and target recognition,” in Proc. Computational Imaging V at SPIE Electronic Imaging, 2007.
  • [22] M. A. Davenport, P. T. Boufounos, M. B. Wakin, and R. G. Baraniuk, “Signal processing with compressive measurements,” IEEE Journal of Selected Topics in Signal Processing, vol. 4, no. 2, pp. 445–460, 2010.