Simultaneous Registration of Image Sequences -- a novel singular value based images similarity measure

07/22/2019 ∙ by Kai Brehmer, et al. ∙ Universität Lübeck 0

The comparison of images is an important task in image processing. For a comparison of two images, a variety of measures has been suggested. However, applications such as dynamic imaging or serial sectioning provide a series of many images to be compared. When these images are to be registered, the standard approach is to sequentially align the j-th image with respect to its neighbours and sweep with respect to j. One of the disadvantages is that information is distributed only locally. We introduce an alternative so-called SqN approach. SqN is based on the Schatten-q-norm of the image sequence gradients, i.e. rank information of image gradients of the whole image sequence. With this approach, information is transported globally. Our experiments show that SqN gives at least comparable registration results to standard distance measures but its computation is about six times faster.



There are no comments yet.


page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Fusion and comparison of images is an important task in image processing. Examples include motion correction in Dynamic Contrast Enhanced Magnetic Resonance Imaging (DCE-MRI) or reconstruction in Histological Serial Sectioning (HSS). These tasks require a suitable image similarity measure which takes application dependent image features into account. For example, in DCE-MRI or HSS the measure has to be invariant under intensity variations due to contrast uptake or staining artefacts, respectively.

For pairs of two images, a variety of options has been proposed and is well-understood. Among the various choices are 

based measures, normalized gradient fields (NGF), mutual information or Kullback-Leibler divergence; see, e.g. 

[5, 6] and references therein. For scenarios where more than two images , , are to be aligned, the standard approach is to adapt the pairwise procedure in a sequential fashion. More precisely, the -th image is aligned with respect to the neighbours and  for all . Since the correct aligned neighbours are yet unknown, the procedure has to be repeated until convergence.

This paper proposes an alternative so-called  approach, which is based on the rank of image gradients of the whole sequence. A key feature of this new approach is that similarity information is made globally. The approach is inspired by work of Möllenhoff et. al. [7] and Haber et.  al. [4]. In [7], Schatten--Norms are used for color image denoising and in [4] local normalized gradient fields are introduced as for pairwise image registration. In our paper,  is used as a data fitting term and globally normalized image gradients of the whole sequence with arbitrary many images are used as a starting point.

We derive and motivate the  similarity measure. We also compare the performance to state-of-the-art measures on real life data. Our computations for a serial sectioning of a mouse brain indicate that  results are qualitatively comparable to a sequential based registration but can be obtained about six times faster; see also [2].

2 The novel similarity measure SqN

The basic idea of our novel measure derives from color image denoising [7]. There, the linear dependency of the gradients of the three color channels is used for regularization. This dependency is quantified by a Schatten--Norm [1]. We generalize this idea and use it in the context of image registration. In particular, we extend the idea to an arbitrary number of images. Moreover, we also use normalized gradient fields rather than image gradients. Finally, we use the functional as a datafit rather than a regularizer.

The Schatten--Norm of a matrix is essentially the 

-norm of the vector of its singular values 

[1]. More precisely, for any 

there exists a singular value decomposition (SVD) 

[3], where  denotes the -by-identity matrix, is a diagonal matrix with diagonal entries and . The Schatten--(quasi)-norm of  is then defined as

We assume that  dimensional images  are given, where  and the spatial dimension is denoted by . Following [4], regularized normalized gradients are given by

Here, the parameter  discrimiantes signal from noise; see [4] for a discussion of choices of  and discretization issues. Setting  we define the new image similarity measure:

The registration model is to minimize , with  and ; cf. [6].

axial coronal sagittal


Figure 1: Registration results for a histological serial sectioning of mouse brain; data courtesy of O. Schmitt, University of Rostock, Germany. Representative axial, coronal, and sagittal slices of the 3D data of size 256-by-256-by-189 are shown. Displayed are linearly pre-aligned data (top row), -registered data (middle row), and -registered data (bottom row).

3 Numerical Results and Discussion

We present results for a histological serial sectioning of a sectioned mouse brain, data courtesy of Oliver Schmitt. For results on DCE-MRI data we refer to [2]. Fig. 1 shows results for a sequential linear pre-registration, the new  based registration, and a standard sequential based registration as a reference (robust and fast to compute).

Although only one iteration was performed for the sequential  registration, the computing time is about six times as for the  approach. The  result also shows a much stronger spatial correlation, indicating that the sequential approach has not yet converged.

Our future work addresses the optimal choice of  (currently  as in [7]) and the extension to 3D.


The authors acknowledge the financial support by the Federal Ministry of Education and Research of Germany in the framework of MED4D (project number 05M16FLA)


  • [1] Bhatia, R., Matrix analysis, Vol. 169, Springer Science & Business Media, 2013.
  • [2] Brehmer, K., Wacker B., Modersitzki J., A Novel Similarity Measure for Image Sequences, International Workshop on Biomedical Image Registration (WBIR), pp. 47–56. Springer, Cham, 2018.
  • [3] Golub, G. H., Van Loan, C. F., Matrix computations, 3rd, Johns Hopkins Univ. Press, Baltimore, MD, USA, 2012.
  • [4] Haber, E., Modersitzki, J., Beyond Mutual Information: a simple and robust alternative, Bildverarbeitung für die Medizin, 350-354, 2005.
  • [5] Modersitzki, J., Numerical methods for image registration, Oxford University Press, 2004.
  • [6] Modersitzki, J., FAIR: flexible algorithms for image registration, SIAM, 2009.
  • [7] Möllenhoff, T., Strekalovskiy, E., Möller, M., Cremers, D., Low rank priors for color image regularization

    , International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition. Springer, Cham, 2015.