A Subpixel Registration Algorithm for Low PSNR Images

03/31/2018 ∙ by Song Feng, et al. ∙ 0

This paper presents a fast algorithm for obtaining high-accuracy subpixel translation of low PSNR images. Instead of locating the maximum point on the upsampled images or fitting the peak of correlation surface, the proposed algorithm is based on the measurement of centroid on the cross correlation surface by Modified Moment method. Synthetic images, real solar images and standard testing images with white Gaussian noise added were tested, and the results show that the accuracies of our algorithm are comparable with other subpixel registration techniques and the processing speed is higher. The drawback is also discussed at the end of this paper.



There are no comments yet.


page 2

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Many image processing applications require registering multiple overlaying the same scene images taken at different times, such as image fusion and super-resolution in astronomical observation, remote sensing, biomedical imaging and so on. Most astronomical objects are extremely faint and very low contrast and emit light in certain specific wavelengths. These characteristics create many problems in image processing, especially in images registration for increasing spatial solution or signal-to-noise ratio by accumulating multiple frames. For example, hundreds of weak solar Magnetic Field images are needed to be registered for decreasing the effects of atmospheric turbulence, wind and the tracking accuracy of the telescope itself in real-time. Due to the short time interval of images, the peak signal-to-noise ratio (PSNR) is less than 20dB usually.

A wide variety of algorithms can be found for solving image registration problem. The techniques can be classified in feature-based and area-based (or intensity-based)

[1]. Feature-based techniques aim to detect the pairwise spatial relation or various descriptors of features, which represented by the control points (CPs), such as points themselves, end points or centers of line, centers of gravity of regions and so on. This approach can handle complex between-image distortions, but requires that the register images must have many prominent details. However, the edges of solar image texture are fuzzy and smooth. CPs for feature-based techniques are hard to find.

The basic idea of area-based matching techniques is to achieve minimizing the compared images’ squared error, maximizing their correlation without any structural analysis. Typical examples include methods using image correlation functions, which can be calculated in spatial or frequency domain. Over the years, various high-accuracy image registration algorithms based on correlation techniques have been developed, especially in processing subpixel image registration.

Usually, subpixel registration covers estimate of subpixel translation, rotation and scaling

[2]. In this paper, we are primarily concerned with evaluation of 2D rigid translation and propose a fast subpixel shift registration algorithm based on image correlation for low PSNR images.

Cross-correlation (CC) and phase-correlation (PC) algorithms are two well-known images registration techniques in frequency domain[1],[3]

. Both of them take Fourier Transform (FT) first and calculate the cross-power or phase of the cross-power spectrum, then take Inverse Fourier Transform (IFT) to generate correlation surface. The coordinate of the maximum on the surface can be used as an estimate of the horizontal and vertical components of translation between two images. Comparing with CC, PC uses the normalized cross-power spectrum instead of cross-power spectrum, so it provides a distinct sharp peak at the point of registration whereas CC yields several broad peaks

[1]. If pixel level registration is satisfactory for applications, locating the PC peak is equivalent to finding the displacement at pixel level.

In order to obtain subpixel accuracy and keep using the same algorithms of locating the peak coordinate of PC, traditional approach is to compute an upsampled correlation between the shifted image and reference image. This approach not only significantly increases the size of IFT that need be computed as the required accuracy increases (for example, registration within 1/100 of a pixel requires computational cost 10000 times larger than original images being correlated), but also strongly associates with interpolation techniques

[4]. Literature[5] proposed a more efficient implementation using nonlinear optimization algorithm to reduce computational time and memory requirement. Literature[6] discussed different solutions to address the interpolation problem in image registration based on high degree B-spline. Since the signal power in the cross or phase correlation is not concentrated in a single peak, but rather in several coherent peaks mostly adjacent to each other, the approaches of estimating the peak position on subpixel level without upsampling images were developed during last ten years. In phase correlation, the peak is shape and yields a 2D Dirichlet kernel, which can be closely approximated by a function. Some different fitting or interpolation functions in a given neighborhood were proposed by [7]-[10], such as fitting of triangle, 1-D or 2-D Gaussian or functions to achieve high accuracy subpixel registration. Base on the Fourier Shift Theorem, the shift parameters can be computed by slope of phase difference of two images in the Fourier domain. Several algorithms were proposed by [11]-[14] as well. And in the literature[15], a method based on gradient correlation(GC) was presented and compared with PC. Literature[11],[16] evaluated the subpixel registration accuracies of different phase-based method.

Ii Proposed Algorithm

For achieving high accuracy result, our method is based on Coarse-to-Fine approach[17]. In the first stage, a coarse pixel level displacement is obtained and image is shifted in pixel level. In the second stage, a finer approximation is performed to assess subpixel translation. Since pixel level registration is quite simple and easy implemented, the refinement of coarsely registered images to subpixel accuracy is our focus in this paper.

Although popular, phase correlation techniques have some disadvantages. It is more sensitive to noise than direct CC, both for low-pass and high-pass inputs. Sharpening the correlation peaks comes at the expense of increased sensibility to noise of the computed maximum position[3]. Barbara Zitova et al.[1] indicated that in a general case PC could fail in case of multimodal data as well. The peak of cross correlation function is rather broad, and there are more surrounding pixels could be involved to estimate the displacement instead of a few of pixels in PC surface. Therefore, cross correlation is selected as our basic approach.

In our application, since low computational complexity is critical, we try to avoid any fitting or interpolation, and calculate centroid of peak directly in our algorithm. In astronomical image processing, Modified Moment is a popular digital centering algorithm[18] and has been widely applied in measurement of star positions in CCD images. Comparing with classic Gaussian fit, it can provide same accuracy with very high speed.

In this method, a threshold level is set and only those count levels in the data array that are above this threshold are considered in the centering process. Specifically, once the threshold is determined in the data array, each pixel in the array is thresholded according to the following precepts:


Then, the centroid of the peak is set equal to the first moment of the marginal distribution.


Single star peak looks like cross correlation surface, which is surrounded by a few of coherent peaks. We apply Modified Moment in the estimate of centroid of cross correlation surface. First, select a radius around the maximum of the peak, and set the threshold by the minimum value inside the circle. Second, calculate each pixel value above the threshold by Eq.(1). Third, determine the centroid with the first moment by Eq.(2) and Eq.(3). The centroid coordinates equals the displacement between two images.

Obviously, the algorithm is based on the assumption that the maximum of the peak is at same position as the centroid. It means the correlation value above the threshold should be symmetrical in the horizontal and vertical directions. The assumption is acceptable in the small center area around the cross correlation peak. In our experiment, the radius was set to 3 pixels.

Iii Experimental Setup

In order to test the performance of the accuracy and robustness against noise of our algorithm, two sets of images with subpixel translation were generated without any interpolation technique.

In date set I, the reference image and shifted images were generated by analytical expressions, and they could be calculated by a liner combination of several two-dimensional Gaussian functions.

Fig. 1: The images of data set I and II were used in algorithm performance testing. (a) Synthetic image. (b) 656 nm wavelength of solar images observed on the earth. (c) 532 nm wavelength of solar images observed on the earth. (d) 17 nm wavelength of solar images observed on the space.
Fig. 2: Results of four algorithms on the test images which are shown on Fig.1, where X axis represents the PSNR in dB and Y axis represents the accuracy of registration on specific PSNR. (a) Synthetic image. (b) 656 nm wavelength of solar images observed on the earth. (c) 532 nm wavelength of solar images observed on the earth. (d) 17 nm wavelength of solar images observed on the space.

Here the coefficient is the amplitude, and are the center and

is the variance of each Gaussian function. Two hundred Gaussian functions were applied in our synthetic image, and their

, and

were generated by uniform distribution on the interval [0, 128] and [1, 6]. The image is shown on Fig.1(a).

and define the applied transformation corresponding to the shift through the and directions, which are chosen from a uniform distribution on the interval [-0.5, 0.5]. The advantage of this method is that an analytical translation field can be straight forward applied to deform reference image[17]. One hundred images of 128128 were generated with different subpixel translation.

Fig. 3: Results of four algorithms on the standard testing images, where X axis represents the PSNR in dB and Y axis represents the accuracy of registration on specific PSNR. (a) Camera Man (b) Lena (c) Pentagon (d) Barbara

In data set II, we followed the scheme described in [14],[16], the reference and register images were generated by filtering and downsampling a single high-resolution image 22002200. The downsampled images were shifted with respect to each other by integer amounts and in the high-resolution grid. After downsampling by a factor D of in each dimension, the relative shifts became fractional shifts of size. After selecting a reference image, all the others were a complete set of translated versions of the reference image within [-0.5, 0.5] pixel in both direction, with exact subpixel displacements in portions of 1/D. We chose D=16, and generated 289 images of 128128. Three real typical solar images shown on Fig.1 (b)-(d) were employed as testing images, which were observed on different times, locations and wavelengths.

Then the white Gaussian noises with different variances were added to both reference and shifted images, and the variances of PSNR were from 10dB to 40dB on the interval of 2dB.

Iv Results and Discussion

For testing the performance, our proposed method applied to data set I and II, and three different subpixel registration algorithms that were proposed by M. Guizar-Sicairos et al.[5] based on upsampled cross correlation, by P.Vandewalle et al.[11] based on the slope of the phase difference in frequency Domain, and by D.Keren et al.[19] based on Taylor expansions in spatial domain were applied as well.

Fig.2 shows the accuracies of four algorithms on the test images versus PSNR, where X axis represents the PSNR in dB and Y axis represents the accuracy of registration on specific PSNR. Since the real subpixel level translations of both data sets were known exactly, the accuracies of different algorithms could be estimated by standard deviation of their results. The conclusion is that our proposed algorithm provides excellent performance both for data set I and II, and the approach by D.Keren et al. also shows very good results except image Fig.1(b).

Besides data set I and II, standard testing images were processed too, include Barbara, Camera Man, Lena and Pentagon. And the results are shown on Fig.3.

Except image Lena, the proposed algorithm provides acceptable subpixel accuracy compared with other algorithm especially in low PSNR. However, the results also show that the performance depends on the autocorrelation of images. If the cross correlation surface is too flat, it is hard to get high accuracy of subpixel registration. Fortunately, in our application, autocorrelation of the images is low.

The calculation speed of our proposed algorithm is higher than others. Under our experiment environment (Lenovo X200s notebook with 1.86GHz Intel Core2 Duo CPU, 4G memory, Windows 7 and Matlab 7.0), the measurement of the subpixel translation between two 128128 images took 5 milliseconds by our code, and 66 milliseconds by M.Guizar-Sicairos et al., 8 milliseconds by P.Vandewalle et al., and 152 milliseconds by D.Keren et al. This is important for us, because we would like to get real-time image registration.

V Conclusions

The methodologies we describe in this paper consist of three steps. First, calculate the cross correlation surface. Second, set the threshold by the minimum value inside a small circle around the maximum of the peak. Third, estimate the centroid above the threshold by the Modified Moment method. Based on the results of processing both synthetic image and real images, our algorithm presents comparable high accuracy for noisy image subpixel registration. Neither upsample nor interpolation techniques are involved, so the speed of the proposed algorithm is very high and could be implemented in real-time image processing. The drawback of our algorithm is that we might not get the prefect result if the autocorrelation of image is high, since cross correlation technique is applied. Obviously, the performance depends not only on PSNR, but also on the construction of images. It is hard to develop an algorithm which could provide perfect accuracy for all images. Considering of our application, the proposed algorithm is a better solution than others.

Vi Acknowledgments

We appreciate the support from National Natural Science Foundation of China (11163004, 11003041), and Open Research Program of Key Laboratory of Solar Activity of Chinese Academy of Sciences (KLSA201205, KLSA201221).


  • [1] B. Zitova and J. Flusser, “Image Registration Methods: a Survey”, Image and Vision Computing, Vol.21, No. 11, pp. 977-1000, October 2003.
  • [2] B.S.Reddy and B. Chatterji, “An FFT-Based Technique for Translation, Rotation, and Scale-Invariant Image Registration”, IEEE Transactions on Image Processing, Vol. 5, No. 8, pp. 1266-1271, August 1996.
  • [3] R. Manduchi and G.A. Mian, “Accuracy Analysis for Correlation-Based Image Registration Algorithms”, in Proc. ISCAS, Chicago, 1993, pp. 834-837.
  • [4] Q.Tian and M.N. Huhns, “Algorithms for Subpixel Registration”, Computer Vision, Graphics, and Image Processing, Vol. 35, No. 2, pp. 220-233, August 1986.
  • [5] G.S. Manuel, S.T. Thurman,J.R. Fienup, “Efficient Subpixel Image Registration Algorithms”, Optics letters, Vol.33, No. 2, pp. 156-158, 2008.
  • [6] G. K. Rohde, A. Aldroubi, D.M. Healy, “Interpolation Artifacts in Sub-pixel Image Registration”, IEEE Transactions on Image Processing, Vol. 18, No. 2, pp. 333-345, February 2009.
  • [7] V. Argyriou, T. Vlachos, “On the Estimation of Subpixel Motion Using Phase Correlation”, Journal of Electronic Imaging, Vol. 16, No. 3, pp. 033018, July 2007.
  • [8] H.Foroosh, J. B. Zerubia, M. Berthod, “Extension of Phase Correlation to Subpixel Registration”, IEEE Transactions on Image Processing, Vol. 11, No. 3, pp. 188-200, August 2002.
  • [9] K.Takita, T.Aoki, Y.Sasaki, T. Higuchi, K. Kobayashi, “High-accuracy Subpixel Image Registration Based on Phase-only Correlation”, IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences E Series A, Vol.E86-A, NO.8, pp. 1925-1934, August 2003.
  • [10] R.J. Althof, M.G.J. Wind, J.T. Dobbins III, “A Rapid and Automatic Image Registration Algorithm with Subpixel Accuracy”, IEEE Transactions on Medical Imaging, Vol. 16, NO.3, pp. 308-316, June 1997.
  • [11] P. Vandewalle, S. Susstrunk, M. Vetterli, “A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution”, EURASIP Journal on Applied Signal Processing, pp. 233-233, January 2006.
  • [12] M. Balci, H. Foroosh, “Subpixel Registration Directly From the Phase Difference”, EURASIP Journal on Applied Signal Processing, pp. 1-11, 2006.
  • [13] W.S. Hoge, “A Subspace Identification Extension to the Phase Correlation Method”, IEEE Transactions on Medical Imaging, Vol. 22, NO. 2, pp. 277-280, February 2003.
  • [14] H. S. Stone, M. T. Orchard, E. C. Chang, S. A. Martucci, “A Fast Direct Fourier-based Algorithm for Subpixel Registration of Images”, IEEE Transactions on Geoscience and Remote Sensing, Vol. 39, NO. 10, pp. 2235-2243, October 2001.
  • [15] G. Tzimiropoulos, V. Argyriou, T. Stathaki, “Subpixel Registration with Gradient Correlation”, IEEE Transactions on Image Processing, Vol. 20, NO. 6, pp. 1761-1767, June 2011.
  • [16] E. Vera, S. Torres, “Subpixel Accuracy Analysis of Phase Correlation Registration Methods Applied to Aliased Imagery,” in EUSIPCO, Lausanne, Switzerland, 2008.
  • [17] A. M. R. Sousa, J. Xavier, M. Vaz, J. J. L. Morais, V. M. J. Filipe, “Cross-Correlation and Differential Technique Combination to Determine Displacement Fields”, Strain, Vol.47, pp. 87-98, 2011.
  • [18] R.C. Stone, “A Comparison of Digital Centering Algorithms”, The Astronomical Journal, Vol. 97, NO. 4, pp. 1227-1237, April 1989.
  • [19] D. Keren, S. Peleg, R. Brada, “Image Sequence Enhancement Using Sub-pixel Displacements”, In

    Proceedings of IEEE Conference on Computer Vision and Pattern Recognition

    , Ann Arbor, MI, 1988. pp. 742-746.