3D/4D ultrasound registration of bone

by   Jonathan Schers, et al.

This paper presents a method to reduce the invasiveness of Computer Assisted Orthopaedic Surgery (CAOS) using ultrasound. In this goal, we need to develop a method for 3D/4D ultrasound registration. The premilinary results of this study suggest that the development of a robust and "realtime" 3D/4D ultrasound registration is feasible.


page 1

page 2

page 5


Global Multi-modal 2D/3D Registration via Local Descriptors Learning

Multi-modal registration is a required step for many image-guided proced...

Towards segmentation and spatial alignment of the human embryonic brain using deep learning for atlas-based registration

We propose an unsupervised deep learning method for atlas based registra...

The Ultrasound Visualization Pipeline - A Survey

Ultrasound is one of the most frequently used imaging modality in medici...

Similarity Registration Problems for 2D/3D Ultrasound Calibration

We propose a minimal solution for the similarity registration (rigid pos...

Lightweight Residual Network for The Classification of Thyroid Nodules

Ultrasound is a useful technique for diagnosing thyroid nodules. Benign ...

3D B-mode ultrasound speckle reduction using deep learning for 3D registration applications

Ultrasound (US) speckles are granular patterns which can impede image po...

A Feature-Driven Active Framework for Ultrasound-Based Brain Shift Compensation

A reliable Ultrasound (US)-to-US registration method to compensate for b...

I Introduction

The Development of 3D ultrasound offers interesting prospects for surgical navigation. Indeed, this non-invasive data and “realtime” imaging technology could reduce the invasiveness of Computer Assisted Orthopaedic Surgery (CAOS) by replacing the optical trackers which, when pinned into the bones (Fig. 1), could increase the risk of infection and cause extra pain owing to extra incisions and exposed bone surfaces. This implies the development of a method for rigid registration in order to track a bony structure. This method has to be at the same time robust and “realtime”.

However, due to the speckles, the shadowing effects and the poor quality of the images, the registration of this imaging modality is a challenging process. Consequently, the research on intra-modality registration of ultrasound images is scarcely reported in the literature [1], [2], but there appears to be a greater interest in work related to the application of ultrasound images [3].

To our Knowledge there is no report on the evaluation of a method for tracking bone structures whitout invasive trackers.

Fig. 1: Optical trackers which are rigidly pinned in to the bone during an anterior cruciate ligament surgery.
Fig. 2: 4D ultrasound image of an anterior tibial tuberosity, i.e. 3 orthogonal planes obtained in “realtime”.

This article is organized as follows. Section 2 presents the material and the method. Evaluation and results are presented in section 3. Section 4 concludes the article and deals with the future work.

Ii Material and Method

Ii-a Material

Ultrasound data have been obtained with a General Electric Voluson 730 Pro, equipped with a 3D RSP 6-16 MHz probe. The size of the ultrasound volumes is voxels, and the voxel size is 0.28 mm. Each volume acquisition take almost 5s.

Ii-B Method

The essential idea of the method that we propose consists in producing a coarse segmentation of the interface between the bone and the soft tissues to select a region of interest around the structures containing information.

Ii-B1 Automatical coarse segmentation

Several types of information, based on the physics of the ultrasound imaging – (1) the bone appears to be hyper-echoic because the difference between the acoutic impedance of the bone and the soft tissues is important, (2) due to the bones have a high absorption rate and there is no imaging possible beyong them, an acoustic shadow is found behind the interface and (3) only the more or less horizontal interfaces can be seen in images because the reflexion of ultrasound is almost specular –, have been used in four different steps. The differents steps have been summarized in the Fig. 3.

In a first step, the images have been binarized according to Otsu’s thresholding method which allows a good approximation of the separation between the echogenic zone and the shadow zone


In a second step, a horizontal sobel filter has been applied to detect interfaces. We have decided to use this filter because the bone interfaces which can be found are nearly horizontal.

In a third step, the images have been averaged to remove the noise. And, eventually, fusioning has been used relying on two conditions:

  • the pixel value is greater than Otsu’s threshold;

  • the pixel belongs to an interface (i.e. it is a local maximum on the ultrasound beam).

A dilatation has also been performed to increase the number of selected pixels. The results of this processing is a region of interest containing information.

Fig. 3: Image processing to obtain the automatical coarse segmentation.

Ii-B2 Registration

The performance of using a voxel-based registration for image registration has been recognized in the literrature [5]. Consequentely, and because to obtain an accurate segmentation of the bone in ultrasound imaging for a surface-based registration is a challenging process, a voxel-based registration approach has been used.

The images have then been compared according to the normalised cross-correlation (NCC) similarity measure. We have chosen this similarity measure because in the literature [6], this measure seems to be the best for monomodality registration. represents the mean value of the image and represents the mean value of the image . The NCC is defined by as follow:

Among the optimization algorithms, we have selected the simplex method of Nelder and Mead [7] for its robustness and convergence time that’s why this method is often used for image registration [8], [2], [9], [10], [11] and [6]. This method optimizes all the parameters in the same time. The simplex method of Nelder and Mead consists in updating a -dimensional simplex such as the value of the similarity measure decrease. At each step, the simplex is modified by geometric operations. A new vertex is accepted according to the value of the similarity measure.

The size of the initial simplex in a parameter space is an important step using simplex optimization. For rigid transformation the space is 6-dimensional. Before determining the size of the initial simplex, we need to process a normalization so that the displacement in the parameters space is approximately the same as in the spatial space. This relationship is checked for translation. However, it is not true for rotation, where the displacement of a voxel is dependent on its distance from the axis rotation. In the data sets we have used in this study, a unit parameter corresponded to 1 mm of translation and 1 of rotation.

There are two conditions for the size of the initial simplex:

  • In principle, it should be greater than the unit dimension along each axis so that it does not get stuck in a local minimum;

  • It should not be greater than the capture range, otherwise, the convergence may not occur.

The size in each direction has been initialized between 3 and 5 units.

The initial attitude corresponds to the result of the previous optimization (for each sequence of images) if registration is successful.

Iii Evaluation and Results

Iii-a Evaluation method

Fig. 4: Correlation beetween and the similarity measure found during registration (60 images).

In order to evaluate our method, we simulated 4D data by reslicing orthogonal 2D slices from the 3D ultrasound volumes. This simulated data were translated and rotated in the maximum range of 10mm –except according to the probe’s axis – / 12 –except according to the perpendicular axis of the probe’s axis 6– which represents a known reference transformation . The registration of the 4D data to the reference volume provides a transformation . The composition of and gives a transformation error of the :

From this transformation error

we extract a translation vector and 3 Euler angles which correspond to the error in each direction. It should be noted that

is correlated to the similarity measure found during registration (Fig. 4): when the error is less than 1mm and 1 then is closed to the absolute maximum, i.e. .

So as to assess our results, we have defined 2 criteria allowing to validate a registration:

  • the similarity measure is over 0.95;

  • the errors in translation and rotation are inferior to 1 mm and 1 .

In the following sections, we will maintain that a registation is proved successful if these 2 critera are estabilished.

Iii-B Results

The proposed method has been tested on ultrasound knee images. Three data sets have been used. Each set contains 60 images (i.e. 2 orthogonal slices). The table I shows the results which we have obtained. In our data, the registration that we propose is successful in almost 65% of the cases, what is encouraging for the future work. Furthermore, the average time per registration is almost 10s per images, what lets imagine than “realtime” is possible. Indeed, our code is not optimized.

Data Sets Success (%) Total Time (s)
1 68.3 696
2 63.3 703
3 61.7 729
TABLE I: Percentage of success for each data sets (60 images per data sets)

Iv Conclusion and Future work

This first evaluation of the method shows that rigid registration of 3D/4D ultrasound data works in approximately 65% of the cases in a reasonable time. The average time of a registration is approxymately 10s. These initial results suggest that the development of a robust and “realtime” 3D/4D ultrasound registration method is feasible.

Current work deals with improvment of accuracy and the robustness.

We also need to evaluate and compare our method with a “gold standard”. The “gold standard” will be defined by the optical trackers pinned in to the bones.


  • [1] M. M. J. Letteboer, M. A. Viergever, and W. J. Niessen, “Rigid Registration of 3D Ultrasound Data of Brain Tumours,” in CARS, ser. International Congress Series, H. U. Lemke, M. W. Vannier, K. Inamura, A. G. Farman, K. Doi, and J. H. C. Reiber, Eds., vol. 1256.   Elsevier, 2003, pp. 433–439.
  • [2] R. Shekhar and V. Zagrodsky, “Mutual Information-Based Rigid and Nonrigid Registration of Ultrasound Volumes,” IEEE Transactions on Medical Imaging, vol. 21, no. 1, pp. 9–22, January 2002.
  • [3] J. P. W. Pluim and J. M. Fitzpatrick, “Image registration,” IEEE Trans. Med. Imaging, vol. 22, no. 11, pp. 1341–1343, 2003.
  • [4] V. Daanen, J. Tonetti, and J. Troccaz, “An information fusion method for the automatic delineation of the bone-soft tissues interface in ultrasound images.” in ECCV Workshops CVAMIA and MMBIA, ser. Lecture Notes in Computer Science, M. Sonka, I. A. Kakadiaris, and J. Kybic, Eds., vol. 3117.   Springer, 2004, pp. 218–229.
  • [5] J. Maintz and M. Viergever, “A survey of medical image registration,” Medical Image Analysis, vol. 2, no. 1, pp. 1–36, 1998. [Online]. Available: citeseer.ist.psu.edu/maintz98survey.html
  • [6]

    J. Kim, J. Fessler, K.L.Lam, J. Balter, and R. T. Haken, “A Feasibility Study on Mutual Information based set-up Error Estimation for Radiotherapy,”

    Medical Physics, vol. 28, no. 12, pp. 2507–2517, 2001.
  • [7] J. A. Nelder and R. Mead, “A simplex method for function minimization,” Computer Journal, no. 7, pp. 308–313, 1965.
  • [8] P. Slomka, J. Mandel, D. Downey, and A. Fenster, “Evaluation of Voxel-based Registration of 3D Power Doppler Ultrasound and 3D Magnetic Resonance Angiographic Images of Carotid Arteries,” Ultrasound in Medicine and Biology, vol. 27, no. 7, pp. 945–955, 2001.
  • [9] P. Radau, P. Slomka, P. Julin, L. Svensson, and L. Wahlund, “Evaluation of Linear Registration Algorithms for Brain SPECT and the Erros due to Hypoperfusion Lesions,” Medical Physics, vol. 28, no. 8, pp. 1660–1668, 2001.
  • [10] F. Maes, D. Vandermeulen, and P. Suetens, “Comparative Evaluation of Multiresolution Optimization Strategies for Multimodality Image Registration by Maximization of Mutual Information,” Medical Image Analysis, vol. 3, no. 4, pp. 373–386, December 1999.
  • [11] C. Meyer, J. Boes, B. Kim, P. Bland, G. LeCarpentier, J. Fowlkes, M. Roubidoux, and P. Carson, “Semiautomatic Registration of Volumetric Ultrasound Scans,” Ultrasound in Medicine and Biology, vol. 25, no. 3, pp. 339–347, 1999.