Single-participant structural connectivity matrices lead to greater accuracy in classification of participants than function in autism in MRI

05/16/2020 ∙ by Matthew Leming, et al. ∙ University of Cambridge 0

In this work, we introduce a technique of deriving symmetric connectivity matrices from regional histograms of grey-matter volume estimated from T1-weighted MRIs. We then validated the technique by inputting the connectivity matrices into a convolutional neural network (CNN) to classify between participants with autism and age-, motion-, and intracranial-volume-matched controls from six different databases (29,288 total connectomes, mean age = 30.72, range 0.42-78.00, including 1555 subjects with autism). We compared this method to similar classifications of the same participants using fMRI connectivity matrices as well as univariate estimates of grey-matter volumes. We further applied graph-theoretical metrics on output class activation maps to identify areas of the matrices that the CNN preferentially used to make the classification, focusing particularly on hubs. Our results gave AUROCs of 0.7298 (69.71 0.6964 (67.72 0.7037 (66.43 Combining structural and functional connectivities gave an AUROC of 0.7354 (69.40 distinguishable network patterns for functional inputs, but did reveal localized differences between groups in bilateral Heschl's gyrus and upper vermis for structural connectivity. This work provides a simple means of feature extraction for inputting large numbers of structural MRIs into machine learning models.



There are no comments yet.


page 1

page 5

page 6

page 7

page 8

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Voxel-based morphometry (VBM) [66] is a means of detecting structural differences in brain anatomy from T1-weighted MRI across groups. In VBM, images are registered to the same coordinate space and segmented into grey matter, white matter, and CSF volumes, before comparisons are made across voxels or groups of voxels using standard statistical tests. Due to its robustness and effectiveness, VBM has enjoyed significant popularity since it was first introduced [67, 4]. Structural covariance networks [40] correlate tissue volumes estimated by VBM in regions across groups of participants to describe relationships that are interpreted as measures of structural integrity or developmental coherence of the brain.

While there have been several cross-sectional findings of structural brain differences in autism [53, 61, 45], these have not been substantiated by a larger-scale analysis [21]. Indeed, characterizations of brain structure in autism have been inconsistent across studies of small sample sizes, although differences at different ages may explain some of this variation [11]; for instance, increased amygdala volumes have been reported in children with autism [60, 55], but not adults [61]. A meta-analysis of VBM studies in autism found disturbance of brain structure in the lateral occipital lobe, the pericentral region, the right medial temporal lobe, the basal ganglia, and proximate to the right parietal operculum [46]. Small-scale studies in children with autism have found altered structural covariance in areas associated with sensory, language, and social development. Altered structural covariance has been found between sensory networks, the cerebellum, and the amygdala in autism [9]. In children, McAlonan et al [39] found that structural covariance indicated localized reductions within fronto-striatal and parietal networks and decreases in ventral and superior temporal grey matter, suggesting abnormalities in the anatomy and connectivity of limbic–striatal (i.e., social) brain system. Language ability correlated with cortical structure and covariance [58], and associations with language development are further supported by studies showing abnormal development of the Heschl’s gyrus [52], an area where functional activation has been associated with development of ‘inner speech” [25]. In adults with autism, structural covariance has shown decreased centrality in cortical volume networks [6].

Autism has been consistently associated with differences in brain function [59, 42]. Efforts to find differences in functional connectivity relative to neurotypical control groups have characterised autism as exhibiting under-connectivity, and thus greater segregation of functional areas [29, 12, 33, 5, 27, 65]. Other studies, mostly of children and adolescents, found evidence of over-connectivity in specific areas of the brains of those with autism [10, 13, 14, 15, 43, 44], locating hyperconnectivity to the posterior right temporo-parietal junction [13] and in striatal areas and the pons [14, 15]. One recent review [24] posited that autism is likely characterised by a mix of hyper- and hypo-connectivity traits.

I-a Machine Learning

Machine learning has found multiple applications to the analysis of brain images in recent years, including pre-processing, segmentation, and diagnostics. Of great interest has been whole-brain phenotypic classification, in which MRI data of two or more phenotypes (such as sexes, or a diseased group and healthy controls) are trained and classified with a machine learning algorithm. Such studies most often include four steps: (1) selection of MRI modality and derived features that are sensitive to the problem at hand ; (2) feature extraction, to reduce data dimensionality; (3) inputting features to train a machine learning model with the selected architecture; and (4) classification and interpretation.

MRI feature extraction is most often performed using techniques previously developed in image analysis, and the specific method is dependent on the selected modality and features. For instance, based on a large body of research and predictable dimensionality reduction [8, 35], it is common to use for classification of functional connectivity matrices [41, 32, 1] representing correlations in time-series between pre-defined regions derived from blood oxygenation level-dependent (BOLD) sensitive fMRI. Likewise, to classify diffusion weighted images (DWI) it is common to use structural connectivity matrices representing the number of white matter tracts traversing the brain between specific regions [16, 31, 19].

However, while there exists several consensus methods for deriving connectivities from fMRI [35, 49] and DWI [8] (though this is still an active area of research [56, 48]), there is no published analogous means of connectivity-based dimensionality reduction for T1-weighted structural MRI, even though it is the most common [38] modality available to study. One reason for the lack of common methodology is that reductions from three-dimensional data to network representations with meaningful physiological interpretation are more difficult to produce than reductions of four-dimensional data. In most existing feature extraction methods for T1-weighted MRI, extracted features are typically independent, univariate measures from regions of interest, such as cortical thickness and surface curvature. However, the lack of a connectivity metric leads not only to the loss of spatial encoding seen in network representations, but fewer features overall (i.e., for ROIs, connectivities output features while univariate measurements output ), reducing effectiveness for machine learning.

For this paper, we designed a similarity metric that reduces T1-weighted MRIs to a network representation without an a priori physiological interpretation, then applied it a dataset of autistic individuals and neurotypical controls. We classified these structural connectivity matrices using an ensemble convolutional neural network (CNN) [37], then compared the method to classifications that used fMRI connectivity matrices of the same dataset, as well as estimates of grey matter volumes. We applied this method to an extremely large dataset of participants with autism, representing a disorder for which machine learning classification on both functional and structural data has proven difficult [51, 30, 23, 34].

I-B Machine Learning in Autism

Previous machine learning studies of autism have achieved classification accuracies that widely varied depending on the modality used, sample size, data quality, selected methods, and diagnostic criteria. A recent study [22]

of 106 high-risk infants between 6-12 months linked brain volume overgrowth to the emergence and severity of autism symptoms, using a deep learning algorithm capable of predicting autism with 81% specificity and 88% sensitivity using brain surface information. Another study by the same group

[18] found that autism could be predicted in 59 6-month-old infants with 81.8% sensitivity using functional imaging. In the general population, efforts in single-participant classification of autism from MRI data have had mixed results [2, 7, 47, 28, 26, 51, 63], with studies rarely exceeding 80% classification accuracy [24]. Again, however, this varies substantially by modality and which site data were collected [30]. In a recent study, Eill et al [17] performed a classification on individuals with autism and neurortypical controls using structural MRI, DWI, and fMRI data, finding that features derived from fMRI provided the highest accuracies with an SVM classifier. They did, however, encounter the issue of fMRI feature extraction simply producing more variables than its structural counterparts, offering the machine learning model more information to work with, although attempts were made to mitigate this issue.

I-C Explainable AI

Besides classification accuracy, an important component to machine learning in scientific discovery is interpretability. The need for explainable machine learning models in a clinical setting has previously been discussed [20]. Clinicians need to fully understand the decision-making process of an automated diagnosis if they are to eventually rely on it. AI models that make a linear, understandable decision-making process are called “expert systems”. These often rely on human-readable information, such as the diagnostic history of an electronic health record. However, such systems would not be capable of making use of more complex datasets that are not always human-readable, such as medical images or genetic records.

Deep learning models have been shown to be capable, at least to a degree, of making sense of complex datasets, in a way that an explainable expert system [20] would not, in applications like whole-brain MRI diagnostics [31, 34, 36, 37]). Unlike expert systems, which rely on linear and human-designed decision-making, deep learning models’ decision-making processes are abstracted by their own complexity, a phenomenon generally referred to as the “black box problem”. Because of the need for clinicians to explain their decisions, this would make deep learning models of limited value. There has been great effort in visualising deep learning models in other contexts in the hope of making them explainable. These methods include occlusion, gradient class activation mapping, and activation maximization [68]. While these methods fail to reveal the exact decision-making process used to make classifications, they are capable of showing which parts of the input data are taken into account for the classification. Use of such techniques can make deep learning models more explainable, and thus more useful in an eventual clinical context. But while such methods help explain machine learning models, the full extent of these techniques, and the exact interpretation of any visualization techniques in a scientific context, is still the subject of ongoing research.

I-D Experiments

In the present work, we present a simple method of deriving structural connectivity matrices from T1-weighted MRI. Our method compared the distributions of grey matter in pairs of parcellated areas of T1-weighted MRI. While this method has no specific physiological interpretation, it acted as an effective means of dimensionality reduction that allowed for T1-weighted MRIs to be encoded into a machine learning model. We describe our dataset, including acquisition and preprocessing methods. We validated these data using a machine learning model previously described [37]

and made comparisons to the classification accuracy using the corresponding fMRI connectivity matrices of the same participants, as well as lower-dimensional data consisting of grey matter volumes averaged within regions. Finally, we used the output class activation maps (CAMs), combined with graph theoretical techniques, to understand which parts of the brain the model focused on, and whether simple linear regression models could spot the same qualities in these data. We further describe a means of combining our structural connectivity matrices with functional connectivity matrices in the same machine learning model to yield improved accuracy.

Ii Materials and Methods

Ii-a Dataset

Age Gender
Collection # Subj. # Conn. Rest Task Min Max Mean Stdev Female Male Autism
ABCD 1049 5142 2296 2846 0.42 11.08 10.12 0.69 2474 2668 61
ABIDE 412 412 412 0 6.00 45.00 17.00 7.16 45 367 181
ABIDE II 682 717 717 0 5.22 55.00 14.39 7.39 169 548 350
BioBank 9791 9791 9791 0 40.00 70.00 55.00 7.51 5178 4613 4
NDAR 1050 7958 5531 2427 0.58 55.83 18.71 7.80 3816 4142 930
Open fMRI 1194 5268 820 4448 5.89 78.00 27.12 10.24 2346 2479 29
Total 14178 29288 19567 9721 0.42 78.00 30.72 14028 14817 1555
TABLE I: Statistics for each dataset used.

We used a dataset comprised of 29,288 total instances each with a structural MRI and an functional MRI in both task-activated and task-absent (rest) conditions. Note that that instances were acquired from the same participant. In total, 1555 data points were from participants with autism. These data were drawn from six different databases: OPEN fMRI, the UK BioBank, ABIDE I, ABIDE II, NDAR (minus ABCD), and ABCD (Table I). Covariates of age, gender, task were also compiled.

Ii-B Pre-processing and feature extraction

The full pre-processing pipeline for functional data is described elsewhere [36]. Functional data were preprocessed using SpeedyPP. Data were first skull stripped. Motion was regressed from time series using wavelet despiking [50]. Data were then registered to the stereotatic space of the Montreal Neurological Institute (MNI), after which they were overlaid on the 116-area AAL parcellation. Datasets with greater than 10% regional dropout, or which otherwise failed the SpeedyPP stage, were excluded. The remaining datasets are presented in Table II. functional connectivity matrices were estimated using Pearson correlation on the averaged timeseries within a region.

To estimate grey matter volumes of each area in the AAL parcellation, structural MRI were first skull stripped using tools from the Analysis of Functional Neuroimages (AFNI) toolbox, then registered to MNI space and grey matter values estimated using FSL VBM. Grey matter volume estimations at each voxel were then averaged within the areas of the AAL parcellation, producing a array.

Ii-C Single-Participant Structural Connectivity Matrices

Fig. 1: Illustration of the procedure used to estimate the structural connectivity matrices used in the present study.

Using the same AAL overlays to derive grey matter volumes, we measured the similarity, between two regions by comparing the distributions of nonzero voxel values within each region ( and ), using the following equation:


which is simply the Wasserstein, or Earth-Mover’s distance. This is an ideal metric as it non-parametically compares two statistical distributions, regardless of relative region sizes. While this similarity metric does do away with spatial encoding and thus eliminates crucial information such as curvature, it acts as a comparison of the distributions of grey matter volumes between two areas in an easily understood way, and at a low computational cost. An illustration of this is shown in Figure 1. While this is a similarity metric that implies no unique physiological relationship between areas, we refer to it as a form of “connectivity” in line with the commonly used vocabulary in connectomics.

Ii-D Comparison of functional and structural connectivities

To determine whether functional connectivity and our novel structural connectivity metric shared information in common, we correlated functional matrices from each instance with their corresponding structural matrices, in 10,000 random samples. We then compared these correlations with a null model estimated by correlating random pairings of functional and structural matrices across the collection. This comparison was done by comparing the two sets of 10,000 R values with a t-test, and indicates the amount of common information encoded by both functional and structural connectivities.

Ii-E Machine learning model and training

We classified individuals with autism and neurotypical controls using, separately, structural connectivity, grey-matter volume, and functional connectivity measurements, as well as a model that combined structural and functional connectivities. We employed the model and training scheme described in [37]. This used an ensemble of 300 convolutional neural networks that each scrambled the unique values of input connectivity matrices, losing some spatial encoding information while avoiding biases in output class activation maps (described below).

In building training, test, and validation sets for our models, a multivariate class balancing scheme was used. Equal ratios of autism and neurotypical control participants were enforced, and equal distributions of age, collection, mean framewise displacement of fMRI data, and intracranial volume were maintained in each class. The class balancing scheme divided data into test, training, and validation sets for each model in the ensemble, ensuring participants with multiple functional connectomes were in the same group. Each model was trained on an Adam optimizer for 100 epochs, after which the epoch with the highest accuracy on the validation set was used. This model then made a prediction on each instance in its test set.

An ensemble of 300 independent CNN models was used to make predictions on the same test set, and an AUROC derived by averaging across instances. When adding models to the ensemble, the AUROC from the aggregated models increased in a predictable way. The AUROCs from between 20 and 300 models were fit to a logarithmic curve with a hard limit in order to predict the projected highest AUROC possible in the limit of a large number of models.

As a result of forced class balancing, each model in the ensemble used an independent subset of approximately 1600 instances. As an effect of this balancing scheme, data from Open fMRI and the UK BioBank, having few participants overall with a diagnosis of autism, were included only infrequently, while data from ABIDE I and II, ABCD, and NDAR were frequently represented.

In total, four cross-sectional classification tasks were undertaken (Table II), specifically: with structural connectivity, with grey matter volumes; with functional encoding; and by combining structural and functional connectivities.

Ii-F Class Activation Map Analysis

Using the Guided Gradient Class Activation Map (Grad-CAM) algorithm [57], which displays areas of the input data most salient in classification, we measured the class activation of each data point in each model proposed, and then averaged these maps generating a CAM for both structural and functional connectivity, as well as a map for grey matter volume. We correlated the structural and functional CAMs to the measured effect size of differences between autism and neurotypical controls for our connectivity data, as a way to determine the similarity of CAMs to conventional statistics.

Next, we isolated hubs in the CAMs. To do so, we first measured the edge betweenness centrality of each edge in our CAMs. We then grouped these values into different communities by maximising modularity of the edge betweenness values (Brain Networks Toolbox [54]). This procedure identified which hubs were most focused on by the classifier.

Iii Results

Fig. 2: The average structural (left) and functional (right) connectivity matrices.

Iii-a Comparison of functional and structural connectivities

Figure 2 shows the average functional and structural connectivity matrices for a balanced group of autism and neurotypical controls. Correlations of functional and structural connectivity matrices from the same participants suggest a modest negative correlation. Across 10,000 random comparisons, the average value of correlated raw edge values was -0.118 against a null model of -0.108. Subsequent t-tests showed that the values of the direct comparisons and null model test fell under different distributions (p=). This indicates that structural and functional connectivities share only a modest amount of similar information for the same participant.

Iii-B Training

Modality AUROC Accuracy
Structural conn., Function 0.7354 69.3980
Structural conn. 0.7298 69.7062
Function 0.6964 67.7180
Structure (GM vols) 0.7037 66.4228
TABLE II: Respective AUROCs and accuracies of ensemble models on different combinations of data.

Training accuracies and AUROCs are given in Table II.

Fig. 3: Left: Overall AUROCs of each dataset included in the analysis for the structure/function ensemble. Right: Projection of the limit of the structure/function/age ensemble models, given data for ensembles of one to 300 models. Adding independent models ad infinitum would result in a maximum predicted AUROC of 0.745.

Classification resulted in a higher AUROC for structural than functional connectivities: 0.7298 and 0.6964, respectively. Classification on univariate grey matter volumes resulted in an AUROC of 0.7037, outperforming functional classification while underperforming structural connectivity classification, although this might be expected considering its lower dimensionality. Combining structure and function resulted in an AUROC of 0.7354 (Figure 3, left), with a projected upper limit of the AUROC of 0.745 (Figure 3, right).

Iii-C Class Activation Map Analysis

Fig. 4: A comparison of the effect size of differences between raw matrix values between groups and the averaged class activation maps. Most of the edge differences passed a nonparametric statistical significance test. When comparing the CAM matrix and the effect size matrices using either linear or nonparametric correlation, neither had any statistically significant associations with one another.

When comparing the output CAMs to their respective functional and structural effect sizes, no statistically significant correlation was observed, and thus the machine learning model relied very little, if at all, on differences detectable by conventional statistics (Figure 4).

Fig. 5: The structural hubs targeted by the structure/function/age encoding. Shown here are the class activation maps (upper left) as well as the edge betweenness centralities of the map (upper right), after it as been sorted into six different hubs via modularity maximization. The hubs, with labeled areas, are shown in the bottom half. (Middle) The three most distinct hubs revolve around the left Heschl’s gyrus; the right Heschl’s gyrus (and, to an extent, the left Putamen); and the upper vermis. The largest hub, in the bottom left, shows scattered-but-weak emphasis on connections to the right frontal medial orbital gyrus. These connections likely reflect the machine learning model’s use of comparisons of certain areas to others in order to assess the developmental difference of such areas in autism.
Fig. 6: Averaged functional class activation maps and the associated edge betweenness centralities, when divided into communities via modularity maximization. Function does not show the same ultra-localized hubness within particular communities in the way that the structural results do, but emphasis is given to several individual connections throughout.

CAMs for structural and functional connectivities, sorted by different detected communities after edge betweenness centrality was measured, are shown in Figures 5 and 6. Structural CAMs showed five distinct groupings, each with distinct hubs that each centered on one or two localised areas, including the left and right Heschl’s gyrus, the upper vermis, the right frontal-medial orbital gyrus, the right pallidum, and the left putamen. The strongest activations were found in left Heshcl’s gyrus.

Localisation was also found, though less distinctly, in functional hubs, notably the left inferior parietal lobe, the left middle temporal lobe, the left olfactory bulb, and the upper vermis. However, focus on particular hubs was not a distinctive feature.

Fig. 7: Top class activation map value results for the 116-area gray matter density classification, showing the areas most focused on in that classification task. The minimum CAM value (not shown) was 1.3622.

CAMs for grey matter volumes are shown in Figure 7. These results had very little in common with the structural connectivity results, with the strongest five activated areas in the right supplementary motor area, the right middle frontal lobe, the right precentral sulcus, the left insula, and the inferior frontal gyrus triangularis.

Iv Discussion

This study proposed a new feature extraction method for inputting structural MRIs into a network-based machine learning model, as well as applicable analysis methods to detect areas of that were particularly involved in determining the classification. Estimating single-participant structural connectivity matrices from T1-weighted images without supplementary modalities such as DWI or fMRI is uncommon, and, with few exceptions [64] research in this area is relatively undeveloped. In structural covariance, VBM data is used to produce inter-regional relationships at a group level, but this is inapplicable at a single-participant level, which is necessary to make structural MRIs applicable to machine learning models. The proposed method provides a means of doing so and validates it in a practical way.

In the univariate grey matter volume results, the CAMs highlighted the right supplementary motor area, right mid frontal lobe, right precentral sulcus, left insula, right frontal inferior triangularis, left frontal inferior orbital lobe, and the right superior temporal lobe (the top 20 areas are shown in Figure 7). Comparing the CAM emphasis of the grey matter volumes to the meta-analysis of autism VBM studies in [46], which found six areas with consistently altered grey matter volumes, some similarities can be seen, notably in the right superior temporal lobe where grey- and white-matter volume differences in the right medial temporal lobe and the left post central gyrus.

Functional analysis did not reveal a pattern of local hubness characterising structural connectivity differences, but rather focused on specific connections. However, a number of general functional communities were identified (Figure 6). Meta-analyses of studies in functional connectivity differences associated wtih autism have not found consistent differences in the brain, but rather in network-wide measures [24]. The lack of hub emphasis in functional results may be additional evidence of network-wide, rather than localized differences between autism and neurotypical control groups seen in other recent findings [62].

In structural connectivity, three definitive hubs were identified: left Heschl’s gyrus, right Heschl’s gyrus, and the upper vermis. The right pallidum and fronto-medial orbital region also showed relatively strong local hubs, though to a lesser degree. Emphasis of the Heschl’s gyrus is in agreement with recent studies in developmental autism, having been implicated previously as an area that develops atypically in autistic children [52]. Function of the area has been associated with development of “inner speech” [25], indicating a difference in development of language capabilities. Our findings differ in that they found this emphasis in structure and not function, but this may be reflective of the lower variability of differences across a single area in the development of grey matter as opposed to function, which likely varies far more across participants, and age groups. The cerebellum, meanwhile, has consistently been cited as an area of difference between individuals with autism and neurotypical controls during development [11], as well as an area of difference in structural covariance associated with autism [9].

The structural connectivity CAMs resulting from our study revealed an emphasis on a number of distinct and localised areas, and these areas were clarified by use of an edge centrality measurement combined with modularity maximization to isolate hubs. The edge betweenness step was added by necessity to place extreme emphasis on a smaller number of more central edges, and only then could modularity maximisation isolate hubs in a meaningful way (see Figure 5).

Our structural connectivity method’s efficacy with the machine learning model suggests that it encoded practically useful information about brain structure, but the interpretation of what these structural hubs indicate physiologically is more complicated. While some correlation is present (Figure 2

) the functional and structural connectivities show largely different patterns. Furthermore, considering that the method used to estimate structural connectivity was a similarity metric, the emphasis on these hubs was less likely an indication that they were centers of a physiological brain network characterising autism. Because the strength of connections was a comparison of grey matter distributions, it is more likely that connections to the identified hubs were used by the machine learning algorithm as a proxy for detecting subtle changes in the morphology of grey matter within those specific regions. Edges connecting to these structural hubs were probably an indirect indication of differences in grey matter between two areas, and the individual connections themselves would not indicate any special physiological relationship. However, this still means that the hubs themselves were important in characterising autism. This lack of an explicit physiological interpretation of our metric, however, does not detract from its utility in the context of machine learning. This structural connectivity metric may simply be viewed as a way of encoding relative spatial information about the morphology of individual areas of the brain.

The univariate grey matter volume results further complicate interpretation because areas different from the structural connectivity results were emphasised by the CAMs, even though both univariate grey matter volumes and structural connectivities were derived from the same imaging data. This brings up three key points. First, the method of encoding data is important because it presents different types of information to the machine learning model. Structural differences in autism (and likely other phenotypic differences) may vary in different ways that are only apparent under specific methods of encoding, and thus the model may have focused on different areas, depending on which method of encoding was performed. This is important for both interpreting the results in the context of a specific machine learning task and understanding the underlying physiological implications. Second, the emphases presented by Grad-CAM were relative; that is, in analysing the distribution of Grad-CAM values, we saw that the model took all areas into account (Figure 7), although with highest focus on the few areas that seemed to hold more influence in the final classification task. This does not, however, mean that other areas were ignored entirely. Third, because of the higher dimensionality of structural connectivities over grey matter values, it may be the case that the machine learning model assumed information about grey matter volumes from a small number of edges, while information about differences in morphology of other areas (e.g., the left and right Heschl’s gyrus), which were not present in the univariate feature extractions, required emphasis by a greater number of edges; this may be crucial to understanding differences in autism generally, or it may have simply helped the model increase AUROC by a margin of 0.0891 between the univariate and connectivity classification tasks. Stated informally, differences in morphology detected by our structural connectivity matrices were more subtle, and so they required the emphasis of a larger number of edges.

In developing this method, other means of estimating single-participant connectivity matrices from T1-weighted MRI were considered, such as estimating the correlation between different univariate measurements (cortical thickness, curvatures, and so on) of the structural image [56, 48], but this was too computationally intensive for a large dataset. Another method was investigated that involved finding the difference between group structural covariance matrices with and without a certain participant. While classifications on these matrices were successful, the matrices themselves varied to such an extent that the output CAMs were inconsistent. In the end, the proposed method was used because of its simplicity and effectiveness in classification.

It is notable that none of the classification accuracies presented in this paper approached the success required for a clinical diagnosis, which would need to consistently exceed 95% accuracy on a substantially large dataset. A likely reason for the comparatively low accuracy in this study specifically is the large dataset size, which, in the context of whole-brain MRI classification, has previously been associated with a drop in accuracy [30, 3]. Nonetheless, deep learning models are useful in these contexts both as statistical models in and of themselves to study autism, and as building blocks to approach clinical-quality accuracy in the future.

Finally, we combined our structural connectivity metric with functional connectivity raising the final AUROC. This shows that our method does not have to be considered as a replacement for any previous methods, but may be used in combination with them in order to make single-participant classifications more effective.

V Conclusion

The present work offers a means of encoding T1-weighted MRI for use in network-based machine learning models, and with a machine learning classification task we have demonstrated an increase in accuracy in classifying individuals with autism when compared with both functional connectivities and classification of univariate grey matter volumes. Furthermore we presented methods of identified areas emphasized by the machine learning model, demonstrating the importance of data encoding and highlighting complications with interpreting results when the feature extractions have no specific physiological interpretation. While this tradeoff, interpretability for higher accuracy, will likely continue to be an issue in machine learning with scientific data, the effects of data encoding on accuracy point towards feature extraction methods as a future direction of investigation.


  • [1] A. Al-Zubaidi, A. Mertins, M. Heldmann, K. Jauch-Chara, and T.F. Münte (2019) Machine learning based classification of resting-state fmri features exemplified by metabolic state (hunger/satiety). Front. Hum. Neurosci. 13, pp. . External Links: Document Cited by: §I-A.
  • [2] J.S. Anderson et al. (2011) Decreased interhemispheric functional connectivity in autism. Cereb Cortex 21, pp. 1134–1146. External Links: Document Cited by: §I-B.
  • [3] M.R. Arbabshirani, S. Plis, J. Sui, and V.D. Calhoun (2017) Single subject prediction of brain disorders in neuroimaging: promises and pitfalls. NeuroImage 145(Pt B), pp. 137–165. External Links: Document Cited by: §IV.
  • [4] J. Ashburner and K.J. Friston (2000) Voxel-based morphometry: the methods. Neuroimage 11, pp. 805–821. External Links: Document Cited by: §I.
  • [5] M. Assaf et al. (2010) Abnormal functional connectivity of default mode sub-networks in autism spectrum disorder patients. NeuroImage 53, pp. 247–256. External Links: Document Cited by: §I.
  • [6] J.B. Balardin et al. (2015) Decreased centrality of cortical volume covariance networks in autism spectrum disorders. J Psychiatr Res. 69, pp. 142–149. External Links: Document Cited by: §I.
  • [7] P. Barttfeld et al. (2012) State-dependent changes of connectivity patterns and functional brain network topology in autism spectrum disorder. Neuropsychologia 50, pp. 3653–3662. External Links: Document Cited by: §I-B.
  • [8] T.E.J. Behrens, H.J. Berg, S. Jbabdi, M.F.S. Rushworth, and M. Woolrich (2007) Probabilistic diffusion tractography with multiple fibre orientations: what can we gain?. NeuroImage 34, pp. 144–155. External Links: Document Cited by: §I-A, §I-A.
  • [9] G.J. Cardon, S. Hepburn, and D.C. Rojas (2017) Structural covariance of sensory networks, the cerebellum, and amygdala in autism spectrum disorder. Front. Neurol. 27, pp. . External Links: Document Cited by: §I, §IV.
  • [10] L. Cerliani, M. Mennes, R.M. Thomas, A. Martino, M. Thioux, and C. Keysers (2015) Increased functional connectivity between subcortical and cortical resting-state networks in autism spectrum disorder. JAMA Psychiatry 72, pp. 767–777. External Links: Document Cited by: §I.
  • [11] R. Chen, Y. Jiao, and E. Herskovits (2011) Structural mri in autism spectrum disorder. Pediatr Res 69, pp. 63–68. External Links: Document Cited by: §I, §IV.
  • [12] V.L. Cherkassky, R.K. Kana, T.A. Keller, and M.A. Just (2006) Functional connectivity in a baseline resting-state network in autism. Neuroreport 17, pp. 1687–90. External Links: Document Cited by: §I.
  • [13] H.Y. Chien, H.Y. Lin, M.C. Lai, S.S. Gau, and W.Y. Tseng (2015) Hyperconnectivity of the right posterior temporo-parietal junction predicts social difficulties in boys with autism spectrum disorder. Autism Res 8, pp. 427–441. External Links: Document Cited by: §I.
  • [14] S. Delmonte, L. O’Gallagher, E. Hanlon, J. McGrath, and J.H. Balsters (2013) Functional and structural connectivity of frontostriatal circuitry in autism spectrum disorder. Front Hum Neurosci 7, pp. 430. External Links: Document Cited by: §I.
  • [15] A. Di Martino et al. (2011) Aberrant striatal functional connectivity in children with autism. Biol Psychiatry 69, pp. 847–856. External Links: Document Cited by: §I.
  • [16] Y. Dodonova, S. Korolev, A. Tkachev, and D. Petrov (2016) Classification of structural brain networks based on information divergence of graph spectra. 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP) , pp. . External Links: Document Cited by: §I-A.
  • [17] A. Eill et al. (2019) Functional connectivities are more informative than anatomical variables in diagnostic classification of autism. Brain Connectivity 9, pp. . External Links: Document Cited by: §I-B.
  • [18] R.W. Emerson et al. (2017) Functional neuroimaging of high-risk 6-month-old infants predicts a diagnosis of autism at 24 months of age. Sci Transl Med. 9, pp. eaag2882. External Links: Document Cited by: §I-B.
  • [19] A. Frau-Pascual, M. Fogarty, B. Fischl, A. Yendiki, and I. Aganj (2019) Quantification of structural brain connectivity via a conductance model. NeuroImage 189, pp. 485–496. External Links: Document Cited by: §I-A.
  • [20] O. Gottesman, F. Johansson, M. Komorowski, A. Faisal, D. Sontag, F. Doshi-Velez, and L.A. Celi (2019)

    Guidelines for reinforcement learning in healthcare

    Nature Medicine 25, pp. 16–18. External Links: Document Cited by: §I-C, §I-C.
  • [21] S. Haar, S. Berman, M. Behrmann, and I. Dinstein (2016) Anatomical abnormalities in autism?. Cerebral Cortex 26, pp. 1440–1452. External Links: Document Cited by: §I.
  • [22] H.C. Hazlett et al. (2017) Early brain development in infants at high risk for autism spectrum disorder. Nature 542, pp. 348–351. External Links: Document Cited by: §I-B.
  • [23] A.S. Heinsfeld, Franco,A.R., R.C. Craddock, A. Buchweitz, and F. Meneguzzia (2018) Identification of autism spectrum disorder using deep learning and the abide dataset. NeuroImage: Clinical 17, pp. 16–23. External Links: Document Cited by: §I-A.
  • [24] J.V. Hull, L.B. Dokovna, Z.J. Jacokes, C.M. Torgerson, A. Irimia, and J.D. van Horn (2017) Resting-state functional connectivity in autism spectrum disorders: a review. Front Psychiatry 7, pp. . External Links: Document Cited by: §I-B, §I, §IV.
  • [25] R.T. Hurlburt, B. Alderson-Day, S. Kühn, and C. Fernyhough (2016) Exploring the ecological validity of thinking on demand: neural correlates of elicited vs. spontaneously occurring inner speech. PLoS ONE 11, pp. e0147932. External Links: Document Cited by: §I, §IV.
  • [26] T. Iidaka (2015) Resting state functional magnetic resonance imaging and neural network classified autism and control. Cortex 63, pp. 55–67. External Links: Document Cited by: §I-B.
  • [27] T.B. Jones et al. (2010) Sources of group differences in functional connectivity: an investigation applied to autism spectrum disorder. NeuroImage 49, pp. 401–414. External Links: Document Cited by: §I.
  • [28] M. Jung et al. (2014) Default mode network in young male adults with autism spectrum disorder: relationship with autism spectrum traits. Mol Autism 5, pp. 35. External Links: Document Cited by: §I-B.
  • [29] M.A. Just, V.L. Cherkassky, T.A. Keller, and N.J. Minshew (2004) Cortical activation and synchronization during sentence comprehension in high-functioning autism: evidence of underconnectivity. Brain 127, pp. 1811–21. External Links: Document Cited by: §I.
  • [30] G.J. Katuwal, N.D. Cahill, S.A. Baum, and A.M. Michael (2015) The predictive power of structural mri in autism diagnosis. Conf Proc IEEE Eng Med Biol Soc , pp. 4270–4273. External Links: Document Cited by: §I-A, §I-B, §IV.
  • [31] J. Kawahara et al. (2017) BrainNetCNN: convolutional neural networks for brain networks; towards predicting neurodevelopment. NeuroImage 146, pp. 1038–1049. External Links: Document Cited by: §I-A, §I-C.
  • [32] A. Kazeminejad and R.C. Sotero (2019) Topological properties of resting-state fmri functional networks improve machine learning-based autism classification. Front. Neurosci. 12, pp. . External Links: Document Cited by: §I-A.
  • [33] D.P. Kennedy and E. Courchesne (2008) The intrinsic functional organization of the brain is altered in autism. NeuroImage 39, pp. 1877–85. External Links: Document Cited by: §I.
  • [34] M. Khosla, K. Jamison, A. Kuceyeski, and M.R. Sabuncu (2018) 3D convolutional neural networks for classification of functional connectomes. MICCAI 2018 , pp. . External Links: Document Cited by: §I-A, §I-C.
  • [35] K.J. Kriston (2011) Functional and effective connectivity: a review. Brain Connectivity 1, pp. 13–36. External Links: Document Cited by: §I-A, §I-A.
  • [36] M. Leming, J.M. Gorriz, and J. Suckling (2020) Deep learning on brain images in autism: what do large samples reveal of its complexity?. International Journal of Neural Systems , pp. . External Links: Document Cited by: §I-C, §II-B.
  • [37] M. Leming and J. Suckling (2020) Stochastic encoding of graphs in deep learning allows for complex analysis of functional connectivity in autism, gender, and memory tasks. arXiv , pp. . External Links: Document Cited by: §I-A, §I-C, §I-D, §II-E.
  • [38] (2006)(Website) External Links: Link Cited by: §I-A.
  • [39] G.M. McAlonan et al. (2005) Mapping the brain in autism. a voxel-based mri study of volumetric differences and intercorrelations in autism. Brain 128(Pt 2), pp. 268–276. External Links: Document Cited by: §I.
  • [40] A. Mechelli, K.J. Friston, R.S. Frackowiak, and C.J. Price (2005) Structural covariance in the human cortex. J Neurosci 25, pp. 8303–8310. External Links: Document Cited by: §I.
  • [41] R. Meszlényi, K. Buza, and Z. Vidnyánszky (2017) Resting state fmri functional connectivity-based classification using a convolutional neural network architecture. Front Neuroinform. 11, pp. 61. External Links: Document Cited by: §I-A.
  • [42] E. Müller, A. Schuler, and G.B. Yates (2008) Social challenges and supports from the perspective of individuals with asperger syndrome and other autism spectrum disabilities. Autism 12, pp. 173–190. External Links: Document Cited by: §I.
  • [43] M.B. Nebel, A. Eloyan, A.D. Barber, and S.H. Mostofsky (2014) Precentral gyrus functional connectivity signatures of autism. Front Syst Neurosci 8, pp. 80. External Links: Document Cited by: §I.
  • [44] M.B. Nebel et al. (2014) Disruption of functional organization within the primary motor cortex in children with autism. Hum Brain Mapp 35, pp. 567–580. External Links: Document Cited by: §I.
  • [45] T. Nickl-Jockschat et al. (2012) Brain structure anomalies in autism spectrum disorder–a meta-analysis of vbm studies using anatomic likelihood estimation. Hum Brain Mapp 33, pp. 1470–1489. External Links: Document Cited by: §I.
  • [46] T. Nickl-Jockschat et al. (2012) Brain structure anomalies in autism spectrum disorder—a meta-analysis of vbm studies using anatomic likelihood estimation. Hum Brain Mapp 33, pp. 1470–1489. External Links: Document Cited by: §I, §IV.
  • [47] J.A. Nielsen et al. (2013) Multisite functional connectivity mri classification of autism: abide results. Front Hum Neurosci 7, pp. 599. External Links: Document Cited by: §I-B.
  • [48] C. Paquola et al. (2019) Microstructural and functional gradients are increasingly dissociated in transmodal cortices. PLoS Biology 17, pp. e3000284. External Links: Document Cited by: §I-A, §IV.
  • [49] A.X. Patel and E.T. Bullmore (2016)

    A wavelet-based estimator of the degrees of freedom in denoised fmri time series for probabilistic testing of functional connectivity and brain graphs

    NeuroImage 142, pp. 14–26. External Links: Document Cited by: §I-A.
  • [50] A.X. Patel et al. (2014) A wavelet method for modeling and despiking motion artifacts from resting-state fmri time series. NeuroImage 95, pp. 287–304. Cited by: §II-B.
  • [51] M. Plitt, K.A. Barnes, and A. Martin (2015) Functional connectivity classification of autism identifies highly predictive brain features but falls short of biomarker standards. NeuroImage Clinical 7, pp. 359–66. External Links: Document Cited by: §I-A, §I-B.
  • [52] M.D. Prigge et al. (2013) Longitudinal heschl’s gyrus growth during childhood and adolescence in typical development and autism. Autism Res 6, pp. 78–90. External Links: Document Cited by: §I, §IV.
  • [53] E. Redcay and E. Courchesne (2005) 2005. Biol Psychiatry. 58, pp. 1–9. External Links: Document Cited by: §I.
  • [54] M. Rubinov and O. Sporns (2010) Complex network measures of brain connectivity: uses and interpretations. NeuroImage 52, pp. 1059–1069. Cited by: §II-F.
  • [55] C.M. Schumann et al. (2004) The amygdala is enlarged in children but not adolescents with autism; the hippocampus is enlarged at all ages. J Neurosci. 24, pp. 6392–6401. External Links: Document Cited by: §I.
  • [56] J. Seidlitz et al. (2018) Morphometric similarity networks detect microscale cortical organization and predict inter-individual cognitive variation. Neuron 97, pp. 231–247.e7. External Links: Document Cited by: §I-A, §IV.
  • [57] R.R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra (2017) Grad-cam: visual explanations from deep networks via gradient-based localization.

    2017 IEEE International Conference on Computer Vision (ICCV)

    , pp. .
    External Links: Document Cited by: §II-F.
  • [58] M. Sharda et al. (2017) Language ability predicts cortical structure and covariance in boys with autism spectrum disorder. Cerebral Cortex 27, pp. 1849–1862. External Links: Document Cited by: §I.
  • [59] T. Simas et al. (2015) Semi-metric topology of the human connectome: sensitivity and specificity to autism and major depressive disorder. PLoS One 10, pp. . External Links: Document Cited by: §I.
  • [60] B.F. Sparks et al. (2002) Brain structural abnormalities in young children with autism spectrum disorder. Neurology 59, pp. 184–192. External Links: Document Cited by: §I.
  • [61] A.C. Stanfield, A.M. McIntosh, M.D. Spencer, R. Philip, S. Gaur, and S.M. Lawrie (2008) Towards a neuroanatomy of autism: a systematic review and meta-analysis of structural magnetic resonance imaging studies. Eur Psychiatry 23, pp. 289–99. External Links: Document Cited by: §I.
  • [62] J. Suckling et al. (2015) A winding road alzheimers disease increases circuitous functional connectivity pathways. Frontiers in Computational Neuroscience 9, pp. . External Links: Document Cited by: §IV.
  • [63] R. Tejwani, A. Liska, H. You, J. Reinen, and P. Das (2017) Autism classification using brain functional connectivity dynamics and machine learning. ArXiv , pp. . External Links: Document Cited by: §I-B.
  • [64] B.M. Tijms, P. Seriès, D.J. Willshaw, and S.M. Lawrie (2012) Similarity-based extraction of individual networks from gray matter mri scans. Cereb. Cortex 22, pp. 1530–1541. External Links: Document Cited by: §IV.
  • [65] S.J. Weng et al. (2010) Alterations of resting state functional connectivity in the default network in adolescents with autism spectrum disorders. Brain Res 1313, pp. 202–214. External Links: Document Cited by: §I.
  • [66] J.L. Whitwell (2009) Voxel-based morphometry: an automated technique for assessing structural changes in the brain. J Neurosci. 29, pp. 9661–9664. External Links: Document Cited by: §I.
  • [67] I.C. Wright et al. (1995) A voxel-based method for the statistical analysis of gray and white matter density applied to schizophrenia. Neuroimage 2, pp. 244–252. External Links: Document Cited by: §I.
  • [68] M.D. Zeiler and R. Fergus (2013) Visualizing and understanding convolutional networks. Computer Vision – ECCV 2014. Lecture Notes in Computer Science 8689, pp. . External Links: Document Cited by: §I-C.