Mapping Tractography Across Subjects

01/29/2016 ∙ by Thien Bao Nguyen, et al. ∙ 0

Diffusion magnetic resonance imaging (dMRI) and tractography provide means to study the anatomical structures within the white matter of the brain. When studying tractography data across subjects, it is usually necessary to align, i.e. to register, tractographies together. This registration step is most often performed by applying the transformation resulting from the registration of other volumetric images (T1, FA). In contrast with registration methods that "transform" tractographies, in this work, we try to find which streamline in one tractography correspond to which streamline in the other tractography, without any transformation. In other words, we try to find a "mapping" between the tractographies. We propose a graph-based solution for the tractography mapping problem and we explain similarities and differences with the related well-known graph matching problem. Specifically, we define a loss function based on the pairwise streamline distance and reformulate the mapping problem as combinatorial optimization of that loss function. We show preliminary promising results where we compare the proposed method, implemented with simulated annealing, against a standard registration techniques in a task of segmentation of the corticospinal tract.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Diffusion magnetic resonance imaging (dMRI) [1]

is a modality that provides non-invasive images of the white matter of the brain. DMRI measures the local the diffusion process of the water molecules in each voxel. That process represents structural information of neuronal axons. From dMRI data, tracking algorithms 

[9, 14] allow to reconstruct the pathways of axons within the white matter of the brain as a set of streamlines, called tractography. A streamline is a 3D polyline representing thousands of neuronal axons in that region of the brain, and a tractography is a large set streamlines, usually .

Current neuroscientific analyses of white matter tractography data are limited to qualitative intra-subject comparisons. Thus, it is quite difficult to use the information for direct inter-subject comparisons [5, 2]. This leads to the need of initial alignment, or registration, of tractographies via some methods before doing further study. Registration is most often performed by applying the transformation resulting from the registration of other images, such as or fractional anisotropy (FA), to tractography [6, 5, 12]. Recently, [10] proposed group-wise registration using the trajectory data of the streamlines. The idea to work on deterministic tractography rather than other images is quite innovative. And, it may be advantageous to directly align the streamlines because the result would be closely related to the final goal of registration.

Similar to [10], in this work, we explore the idea of working on deterministic tractography rather than other images. However, in contrast to all current tractography registration methods, which are based on rigid or non-rigid shape transformation of one tractography into another, our approach tries to find which streamline of one tractography corresponds to which streamline in the other tractography, without transformations. This correspondence is a mapping from one tractography to the other.

In this work we propose to solve the problem of finding the mapping between two tractographies through a graph-based approach similar to that of the well-known graph matching problem [3, 13]. In the graph matching problem the aim is to find which node of one graph corresponds to which node of another graph, under the assumption that graphs have the same number of nodes and that the correspondence is one-to-one.

Given a tractography of streamlines and a distance function between streamlines, we can create an undirected weighted graph by considering each streamline as a vertex and the edge connecting vertex and as the distance between the two streamlines, . Then, intuitively, the problem of tractography mapping becomes very similar to that of graph matching, but with some key differences. Firstly, the size of the two tractographies/graphs is in general not the same. Global differences in the anatomy of the brains, e.g. different volume, motivates this difference. Secondly, in general there is not a one-to-one correspondence between the streamlines/nodes but a many-to-one correspondence. This is anatomically likely if we consider that a given anatomical structure (tract), e.g. the cortico-spinal tract (CST), whose streamlines should have direct correspondence across subjects, may have different thickness, i.e. different number of streamlines. In this case, for example, multiple streamlines of one CST would correspond to a single streamline in the other CST. Because of these differences, it is generally not possible to directly apply efficient graph matching algorithms to the problem of mapping tractographies.

In the following we formally describe the tractography mapping problem starting from the graph matching problem and define the details of the optimization problem to solve. We provide a preliminary algorithmic solution, based on simulated annealing, to minimize the proposed loss function. Then, we apply our proposed solution to a tractography segmentation task in order to compare a standard registration-based method to our proposed method on a fair ground. We conclude the paper with a brief discussion of the preliminary encouraging results.

2 Methods

An undirected weighted graph of size is a finite set of vertices and edges . The graph matching problem can be described as follows. Given two graphs to with the same number of vertices , the problem of matching and is to find the correspondence between vertices of and vertices of , which allows to align, or register, and in some optimal way. The correspondence between vertices of and of is defined as a permutation of the vertices, i.e. there a one-to-one correspondence between the two set of vertices. is usually represented as a binary matrix where is equal to , if the th vertex of is matched to the th vertex of , otherwise . Given and , i.e. the adjacency matrices of the two graphs, the quality of the matching is assessed by the discrepancy, or loss, between the graphs after matching as:


where is the Frobenius norm. Therefore, the graph matching problem becomes the problem of finding that minimize over the set of permutation matrices :


which is a combinatorial optimization problem. The exact solution to this problem is NP-complete and only approximate solutions are available in practical cases [3, 13].

Let and , where is a streamline and , be the tractographies of two subjects. Let be a distance function between streamlines. We define two graphs and with adjacency matrix and where and . Our current choice of is discussed in Section 3, however any common streamline distance from the literature can be used.

The loss function of a mapping from to is then:


where the mapping is a binary matrix and is equal to , if of is mapped to of and otherwise. Note that, in general, is not a permutation matrix, because multiple streamlines can be mapped into the same one. In order to find the optimal mapping , we minimize so that is most similar to :


where is the set of all possible mappings. Because in general and because is a mapping and not just a permutation, the tractography mapping problem has a larger search space than the graph matching problem, i.e. when , is much larger than . As a consequence, the efficient solutions available in the literature of graph matching, e.g. [13], are not applicable, because they heavily rely on the assumptions that we violate here. In Section 3

we implemented a simple preliminary solution to the combinatorial optimization problem by means of the Simulated Annealing meta-heuristic 


2.1 Comparison

In order to compare the proposed method against a standard registration procedure on a fair ground, we cannot rely on the value of the loss function , because it is defined only in the case of mapping. For this reason, we compared the two approaches on the practical task of automatic tractography segmentation, i.e. finding a given tract of interest in given its segmentation in . Our hypothesis is that reducing leads to better overlap between tractographies, which is important for practical applications like segmentation. In Section 3 we describe an experiment to test this hypothesis and provide the necessary details. Here we introduce the metric that we use for comparing registration and mapping. As proposed in [5], we compare the set of voxels crossed by the streamlines of each tractography after mapping or after registration. As measure of the overlap between and 111For sake of brevity we denote as the result of applying mapping to .

, we adopt the Jaccard index:


Note that in the above equation, is the volume computed as number of voxels that any streamline goes through, and indicates the number of voxels in common between and .

3 Experiments

We designed an experiment to provide empirical evidence that reducing the loss in Equation 3 is related to an increase of the Jaccard index, i.e. of the overlap between tractographies.

The dataset used for the experiment is based on dMRI data recorded with a scanner at Utah Brain Institute, gradients ( + ); b-value = ; anatomical scan (). The tractography was reconstructed with the EuDX algorithm [4] using the dipy222 toolbox. We considered 4 healthy subjects and focused the analysis on the corticospinal tract (CST). CST is a set of streamlines projecting from the lateral medial cortex associated with the motor homunculus. This tract is of main interest for the characterization of neurodegenerative diseases, like the amyotrophic lateral sclerosis (ALS). The CST tracts were segmented by the expert neuroanatomists using a toolbox [11] that supports an interactive selection of streamlines. The size of the segmented tracts is reported in Table 1 (see column size).

The reference method, against which we compared mapping, is the affine registration of the tractographies in a common MNI space using the voxel-based FLIRT method [7]. The registration is defined as follows: First, FA images were registered to the MNI-FMRIB-58 FA template, then the affine transformation was applied to the tractographies. The Jaccard index computed between the and in common space is reported in Table 1 (see column FLIRT).

We then used mapping to compute the same quantity. The first step was encoding the tractographies as graphs, which required to define a distance between streamlines. We refer to the commonly used Mean Average Minimum distance (MAM) [14], based on the Hausdorff distance:


where , and .

Mapping a tract such as the CST, which usually comprises streamlines, to an entire tractography , which usually consist of streamlines, is computationally extremely expensive because the space of all possible mappings has size . For this reason, we introduced a heuristic to retain some of the streamlines in . The intuitive idea was to define a superset of streamlines of the CST for subject B, denoted . The heuristic is in two steps: first, we computed the medoid of , and the radius . Second, we filtered the streamlines in such that , where . See Table 1, column , for the actual sizes of the supersets.

Computing the optimal mapping requires to solve, even in an approximate way, the minimization problem of Equation 4. As a preliminary strategy to approximate the optimal mapping , we implemented the simulated annealing (SA) [8] meta-heuristic, a reference method for combinatorial optimization. SA requires the definition of a function to move from the current state, i.e. the current mapping , to a (potentially better) neighbouring one. As transition function we used a stochastic greedy one where, given the current mapping , one streamline of is selected at random and then it is greedily re-mapped to the streamline in providing the greatest reduction in the loss of Equation 3. As starting point of the annealing process, we used the -nearest neighbour of with respect to after the registration of and . We ran the simulated annealing for iterations, which required a few minutes on a standard computer333We are aware that this method of combinatorial optimization can be significantly improved, but we claim that the it was sufficient to do a preliminary investigation of the relation between the loss and the overlap between tractographies, by means of the Jaccard index..

The results reported in Figure 1 show the behaviour of the loss during the optimization process for the mapping of (subject ID ), with respect to the tractography of three other subjects (subject IDs , and ). In all cases, as the number of iterations increases, the value of loss function decreases. In Figure 2 we show an example of experiment with the outcome of FLIRT registration and mapping which refers to subjects 204 and 206. In subfigure A, the source tract is shown in blue, in subfigure B the target tract is show in green and the related superset of streamlines in red. In subfigure C, the result of FLIRT registration is presented, both with respect to the superset on the left and with respect to the target tract on the right. On the right side, it is illustrated the set of streamlines (blue) from the source tract associated to streamlines of target tract . The association between streamlines of and is computed as nearest neighbour after the FLIRT registration. The ratio between blue and green streamlines represents the portion of target tract correctly detected. On the left side of subfigure C, blue streamlines represents the portion of source tract not associated to target tract . In subfigure D, the result of mapping is presented, with the same strategy of presentation of subfigure C. On the right side the visualization shows a greater amount of (blue) streamlines correctly mapped into target tract. Even on the left side the amount of (blue) streamlines erroneously mapped is greater. The sum of blue streamlines on the left and right side represents the portion of streamlines projected from the source to the target. The registration based on FLIRT doesn’t preserve after the alignment the same amount of streamlines from the source tract.

Figure 1: Plots of the normalized loss () as a function of number of iterations with simulated annealing, when mapping the CST of subject 205 to those of subjects 204, 206 and 212.
Figure 2: Example of registration vs. mapping of the corticospinal tract (CST). From the left, the tract to be mapped (subfigure A, in blue), the second tract with its superset (subfigure B, in green, in red), the result of FLIRT affine registration (subfigure C) and of mapping (subfigure D).
A B size Jaccard index
subject ID subject ID FLIRT SA-0 SA-100 SA-1000
205 204 60 124 682 0.18 0.55 0.52 0.59
206 60 100 550 0.15 0.77 0.81 0.82
212 60 68 374 0.10 0.74 0.77 0.90
Table 1: Comparison of registration vs. mapping. The subject IDs of and are reported in the first two columns. Their sizes, together with that of , are in columns three to five. The last four columns report the overlap between and in terms of Jaccard index (higher is better), for FLIRT registration (6th column) and for mapping with simulated annealing at a different number of iterations (SA-0, SA-100, SA-1000 columns).

In Table 1 are reported the results of the comparison between registration and mapping methods, measured by the Jaccard index. The overlap between and provided by FLIRT registration is generally quite poor. This is partly expected because even after the registration of and , and may have a systematic displacement due to the variability of anatomy across subjects. The results of mapping at different iterations of the optimization process shows a remarkable global increase in the Jaccard index and a general trend of improved alignment when more iterations are computed.

4 Discussion and Conclusion

In this work we addressed the challenge of finding an alignment between the tractographies of two subjects. We recast the question as a problem of mapping between two sets of streamlines and we provided the formulation of the corresponding minimization problem. Preliminary results show that this approach is promising despite some limitations. The computational complexity represents a major issue that may prevent to scale up to whole tractography.


  • [1] P. J. Basser, J. Mattiello, and D. LeBihan.

    MR diffusion tensor spectroscopy and imaging.

    Biophysical journal, 66(1):259--267, January 1994.
  • [2] Pierre-Louis L. Bazin, Chuyang Ye, John A. Bogovic, Navid Shiee, Daniel S. Reich, Jerry L. Prince, and Dzung L. Pham. Direct segmentation of the major white matter tracts in diffusion tensor images. NeuroImage, 58(2):458--468, September 2011.
  • [3] D. Conte, P. Foggia, C. Sansone, and M. Vento.

    Thirty years of graph matching in pattern recognition.

    Int. J. Patt. Recogn. Artif. Intell., 18(03):265--298, May 2004.
  • [4] E. Garyfallidis. Towards an accurate brain tractography. PhD thesis, University of Cambridge, 2012.
  • [5] Dan Golding, Marc Tittgemeyer, Alfred Anwander, and Tania Douglas. A comparison of methods for the registration of tractographic fibre images. In P. Robinson and A. Nel, editors, Proceedings of the Twenty-Second Annual Symposium of the Pattern Recognition Association of South Africa, pages 55--59, 2011.
  • [6] Casey B. Goodlett, P. Thomas Fletcher, John H. Gilmore, and Guido Gerig. Group analysis of DTI fiber tract statistics with application to neurodevelopment. NeuroImage, 45(1 Suppl), March 2009.
  • [7] M. Jenkinson and S. Smith. A global optimisation method for robust affine registration of brain images. Medical image analysis, 5(2):143--156, June 2001.
  • [8] P. J. M. Laarhoven and E. H. L. Aarts, editors. Simulated Annealing: Theory and Applications. Kluwer Academic Publishers, Norwell, MA, USA, 1987.
  • [9] Susumu Mori and Peter C. M. van Zijl. Fiber tracking: principles and strategies, a technical review. NMR Biomed., 15(7-8):468--480, November 2002.
  • [10] Lauren J. O’Donnell, William M. Wells, Alexandra J. Golby, and Carl-Fredrik F. Westin. Unbiased groupwise registration of white matter tractography. Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention, 15(Pt 3):123--130, 2012.
  • [11] Emanuele Olivetti, Thien B. Nguyen, and Paolo Avesani. Fast Clustering for Interactive Tractography Segmentation. the 3rd IEEE Intl Workshop on Pattern Recognition in NeuroImaging, 2013.
  • [12] Yi Wang, Aditya Gupta, Zhexing Liu, Hui Zhang, Maria L. Escolar, John H. Gilmore, Sylvain Gouttard, Pierre Fillard, Eric Maltbie, Guido Gerig, and Martin Styner. DTI registration in atlas based fiber analysis of infantile Krabbe disease. NeuroImage, 55(4):1577--1586, April 2011.
  • [13] Mikhail Zaslavskiy, Francis Bach, and Jean-Philippe Vert. A Path Following Algorithm for the Graph Matching Problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(12):2227--2242, October 2008.
  • [14] Song Zhang, S. Correia, and D. H. Laidlaw. Identifying White-Matter Fiber Bundles in DTI Data Using an Automated Proximity-Based Fiber-Clustering Method. IEEE Transactions on Visualization and Computer Graphics, 14(5):1044--1053, September 2008.