Multi-scale Image Fusion Between Pre-operative Clinical CT and X-ray Microtomography of Lung Pathology

02/27/2017 ∙ by Holger R. Roth, et al. ∙ 0

Computational anatomy allows the quantitative analysis of organs in medical images. However, most analysis is constrained to the millimeter scale because of the limited resolution of clinical computed tomography (CT). X-ray microtomography (μCT) on the other hand allows imaging of ex-vivo tissues at a resolution of tens of microns. In this work, we use clinical CT to image lung cancer patients before partial pneumonectomy (resection of pathological lung tissue). The resected specimen is prepared for μCT imaging at a voxel resolution of 50 μm (0.05 mm). This high-resolution image of the lung cancer tissue allows further insides into understanding of tumor growth and categorization. For making full use of this additional information, image fusion (registration) needs to be performed in order to re-align the μCT image with clinical CT. We developed a multi-scale non-rigid registration approach. After manual initialization using a few landmark points and rigid alignment, several levels of non-rigid registration between down-sampled (in the case of μCT) and up-sampled (in the case of clinical CT) representations of the image are performed. Any non-lung tissue is ignored during the computation of the similarity measure used to guide the registration during optimization. We are able to recover the volume differences introduced by the resection and preparation of the lung specimen. The average (± std. dev.) minimum surface distance between μCT and clinical CT at the resected lung surface is reduced from 3.3 ± 2.9 (range: [0.1, 15.9]) to 2.3 mm ± 2.8 (range: [0.0, 15.3]) mm. The alignment of clinical CT with μCT will allow further registration with even finer resolutions of μCT (up to 10 μm resolution) and ultimately with histopathological microscopy images for further macro to micro image fusion that can aid medical image analysis.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Clinical computed tomography (CT) is used for the diagnostic imaging of the living human (in-vivo imaging). As a result, most computational analysis is constrained to the millimeter scale because of the limited resolution of clinical CT. At this millimeter scale, pulmonary blood vessels and lung lobes can be observed. However, finer detailed anatomy is not observable. X-ray microtomograph (CT) on the other hand allows imaging of ex-vivo tissues at a resolution of tens of microns. At this m-scale, the alveoli and bronchiole regions can be clearly observed [1].

The prospective study of pre-operative imaging together with the high-resolution image analysis of resected tissue after surgery may provide us with unique opportunities to verify and potentially improve imaging protocols for cancer diagnostics [2, 3, 4].

In this work, we use clinical CT to image lung cancer patients before partial pneumonectomy (resection of pathological lung tissue). The resected specimen is then prepared for CT imaging allowing the imaging of m-scale anatomy. This study investigates the use of non-rigid intensity-based registration in order to establish a scale-seamless registration between clinical CT and CT, with the ultimate aim of allowing a seamless navigation between anatomical scales inside the human body [1]. Related work is the registration of in-vivo and ex-vivo MRI of surgically resected specimens by [5, 2].

Ii Method

The resected specimen is prepared for CT imaging at a voxel resolution of 50 m (0.05 mm). This high-resolution image of the lung cancer tissue allows further insides into understanding of tumor growth and categorization. For making full use of this additional information, image fusion (registration) needs to be performed in order to re-align the CT image with clinical CT. We developed a multi-scale non-rigid registration approach.

Ii-a Establishing scale-seamless registration between CT and clinical CT

We use a non-rigid registration method to align the CT specimen with the pre-operative clinical CT. This method is driven by the intensity similarity between the images. Any non-lung tissue is ignored during computation of the similarity measure used to guide the registration during optimization. This is achieved via simple thresholding and morphological operations to extract the lung region in clinical CT and CT as pre-processing step.

A coarse-to-fine approach is proposed in order to capture first the largest deformations and then the smaller differences between both images. This is achieved with a four-level multi-resolution pyramid as illustrated in Fig. 1. In order to compensate for the large differences in resolution between the CT (in the case of CT) and clinical CT images, we use a down-sample and up-sample (in the case of clinical CT) pyramid and perform the registration optimizations at each level in a coarse-to-fine fashion.

Fig. 1: Multi-scale image fusion between pre-operative clinical CT and x-ray microtomography of lung pathology in a coarse-to-fine approach.

A good initialization for the registration algorithm is important. We use a few (35) manual point in order to provide affine alignment of the specimen with the pre-operative CT image. After affine initialization, we establish non-rigid alignment using the B-spline registration method, also known as 3D free form deformation based registration of [6] with the implementation provided by [7].

A 3D cubic B-Splines deformation model uses a lattice of control points . The spacing between each control point is uniform and denoted as , , and along the -, -, and -axis respectively. For each voxel in the domain of the target image. the deformation can be computed as:

(1)

where represents the cubic B-Spline function.

The images are aligned by finding the transformation which maximizes the following objective function:

(2)

which combines a similarity measure, , and three penalty constraint terms, , , and . Each term is weighted against each other by user-defined weights , , and .

The similarity measure used between the reference () and floating image () is normalized mutual information (NMI):

(3)

where and the two marginal entropies, and ) is the joint entropy. Its computation requires a joint histogram which is filled by using a Parzen Window (PW) approach [7, 8].

The three constraint terms are used to encourage realistic deformations. The bending energy describes the smoothness of the deformation and is defined as:

(4)

The volume-preserving penalty term discourages large expansions/contractions, and is defined as:

(5)

In addition we prevent the occurrence of folding in the transformation using a folding correction scheme [9]. For each transformed voxel that would cause a negative Jacobian determinant, its influence on its neighborhood control points is computed. The control point positions is then changed until the determinant value is positive.

In an ideal case, the transformations from to (forward) and to (backward) are the inverse of each other, e.g. and [10]. Hence, we include a penalty term that encourages inverse consistency of both transformations. We follow the approach of [11] using compositions of and and add

(6)

The following parameters were found empirically by visual examination of the registration results. We use a four-level multi-resolution pyramid with a maximum of 500 iterations per level. Both the image and B-spline control point grid resolutions are doubled with increasing resolution levels. The final control point spacing between voxels is 5. The objective function weights are set to , , and . These parameters were found to recover the majority of the deformation between the two images, while preventing unrealistic deformations from occurring. We used the open-source software111http://sourceforge.net/projects/niftyreg by [7, 12] for this study.

Iii Results

We collected pre-operative clinical CTs from two patients before partial pneumonectomy. The clinically acquired CT images have dimensions of [512, 512, 435554], and voxel spacings of [0.625, 0.625, 0.6] mm. The CT images used for registration had dimensions of [278512, 278512, 346538] with isotropic voxel spacings of mm. Note that we downsampled the original CT images (circa 50m resolution) by a factor of two for the registration experiments. The tube voltage for CT was 90kVp, and tube current was 110 A.

35 corresponding points where chosen by an expert clinician (SN) in order to provide an initial affine alignment of the specimen with the pre-operative CT image. We then crop the clinical CT image to the extent of the aligned CT image for subsequent non-rigid registration.

Figure 2 shows the alignment of CT lung specimen with pre-operative clinical CT before and after non-rigid registration for one case. A qualitatively better alignment of CT with the target clinical CT regions can be observed.

Fig. 2: Alignment of CT lung specimen with pre-operative clinical CT in axial (a), coronal (b), and sagittal (c) planes: before registration (red), after non-rigid alignment (cyan).

In order to quantitatively evaluate the registration performance, we measure the average minimum surface distance (AvgDist) between the lung surface extracted from clinical CT and CT before and after non-rigid registration. Figure 3 shows the extracted surfaces used for measurement before and after non-rigid alignment. The AvgDist measures are given in Table I. A reduction from 3.3 2.9 (range: [0.1, 15.9]) to 2.3 mm 2.8 (range: [0.0, 15.3]) mm on average can be observed. This is a significant improvement with p 0.001 (Wilcoxon signed rank test).

Fig. 3: Surface alignment of CT lung specimen with pre-operative clinical CT (yellow): before registration (red), after non-rigid alignment (cyan).
AvgDist before registration after registration
[mm] mean std min. max. mean std min. max.
case 1 3.6 3.1 0.0 17.2 2.6 2.8 0.0 15.9
case 2 2.9 2.7 0.1 14.6 2.0 2.7 0.0 14.7
mean 3.3 2.9 0.1 15.9 2.3 2.8 0.0 15.3
TABLE I: Average minimum distances (AvgDist) in mm between the lung surface extracted from clinical CT and CT before and after non-rigid registration.

Iv Conclusions

We presented a method for non-rigid alignment between images of pre-operative clinical CT and x-ray microtomography (CT) of lung pathology. After manual initialization using a few landmark points and affine alignment, several levels of non-rigid registration between down-sampled (in the case of CT) and up-sampled (in the case of clinical CT) representations of the image are performed. This allows us to recover the volume differences introduced by the resection and preparation of the lung specimen. The proposed multi-scale image fusion approach will allow further registration with even finer resolutions of CT (up to 10 m resolution) [13] and ultimately with histopathological microscopy images for further macro to micro image fusion that can aid medical image analysis.

Acknowledgements

This paper was supported by MEXT KAKENHI (26108006 and 15H01116).

References

  • [1] K. Mori, “From macro-scale to micro-scale computational anatomy: a perspective on the next 20 years,” Medical Image Analysis, vol. 33, pp. 159–164, 2016.
  • [2] M. Goubran, C. Crukley, S. de Ribaupierre, T. M. Peters, and A. R. Khan, “Image registration of ex-vivo mri to sparsely sectioned histology of hippocampal and neocortical temporal lobe specimens,” Neuroimage, vol. 83, pp. 770–781, 2013.
  • [3]

    S. Eriksson, S. Free, M. Thom, L. Martinian, M. Symms, T. Salmenpera, A. McEvoy, W. Harkness, J. Duncan, and S. Sisodiya, “Correlation of quantitative mri and neuropathology in epilepsy surgical resection specimens—t2 correlates with neuronal tissue in gray matter,”

    Neuroimage, vol. 37, no. 1, pp. 48–55, 2007.
  • [4] K. Howe, D. Dimitri, C. Heyn, T.-R. Kiehl, D. Mikulis, and T. Valiante, “Histologically confirmed hippocampal structural features revealed by 3t mr imaging: potential to increase diagnostic specificity of mesial temporal sclerosis,” American Journal of Neuroradiology, vol. 31, no. 9, pp. 1682–1689, 2010.
  • [5] M. Goubran, S. de Ribaupierre, R. R. Hammond, C. Currie, J. G. Burneo, A. G. Parrent, T. M. Peters, and A. R. Khan, “Registration of in-vivo to ex-vivo mri of surgically resected specimens: A pipeline for histology to in-vivo registration,” Journal of neuroscience methods, vol. 241, pp. 53–65, 2015.
  • [6] D. Rueckert, L. Sonoda, C. Hayes, D. Hill, M. Leach, and D. Hawkes, “Nonrigid registration using free-form deformations: Application to breast mr images,” IEEE Trans. Med. Imaging, vol. 18, no. 8, pp. 712–721, 1999.
  • [7] M. Modat, G. Ridgway, Z. Taylor, M. Lehmann, J. Barnes, D. Hawkes, N. Fox, and S. Ourselin, “Fast free-form deformation using graphics processing units,” Comput. Meth. Prog. Bio., vol. 98, no. 3, pp. 278–284, 2010.
  • [8] D. Mattes, D. R. Haynor, H. Vesselle, T. K. Lewellen, and W. Eubank, “Pet-ct image registration in the chest using free-form deformations,” IEEE transactions on medical imaging, vol. 22, no. 1, pp. 120–128, 2003.
  • [9] M. Modat, J. McClelland, and S. Ourselin, “Lung registration using the NiftyReg package,” Proc. MICCAI Medical Image Analysis for the Clinic: A Grand Challenge, EMPIRE10, 2010.
  • [10] G. Christensen, X. Geng, J. Kuhl, J. Bruss, T. Grabowski, I. Pirwani, M. Vannier, J. Allen, and H. Damasio, “Introduction to the non-rigid image registration evaluation project (nirep),” Biomedical Image Registration, vol. 4057, pp. 128–135, 2006.
  • [11] W. Feng, S. Reeves, T. Denney, S. Lloyd, L. Dell’Italia, and H. Gupta, “A new consistent image registration formulation with a b-spline deformation model,” in Biomedical Imaging: From Nano to Macro, 2009. ISBI’09. IEEE International Symposium on, pp. 979–982, IEEE, 2009.
  • [12] M. Modat, M. Cardoso, P. Daga, D. Cash, N. Fox, and S. Ourselin, “Inverse-consistent symmetric free form deformation,” Biomedical Image Registration, vol. 7359, pp. 79–88, 2012.
  • [13] K. Nagara, H. Oda, S. Nakamura, M. Oda, H. Homma, H. Takabatake, M. Mori, H. Natori, D. Rueckert, and K. Mori, “Cascade registration of micro ct volumes taken in multiple resolutions,” in International Conference on Medical Imaging and Virtual Reality, pp. 269–280, Springer, 2016.