Investigating Conversion from Mild Cognitive Impairment to Alzheimer's Disease using Latent Space Manipulation

by   Deniz Sezin Ayvaz, et al.
Boğaziçi University

Alzheimer's disease is the most common cause of dementia that affects millions of lives worldwide. Investigating the underlying causes and risk factors of Alzheimer's disease is essential to prevent its progression. Mild Cognitive Impairment (MCI) is considered an intermediate stage before Alzheimer's disease. Early prediction of the conversion from the MCI to Alzheimer's is crucial to take necessary precautions for decelerating the progression and developing suitable treatments. In this study, we propose a deep learning framework to discover the variables which are identifiers of the conversion from MCI to Alzheimer's disease. In particular, the latent space of a variational auto-encoder network trained with the MCI and Alzheimer's patients is manipulated to obtain the significant attributes and decipher their behavior that leads to the conversion from MCI to Alzheimer's disease. By utilizing a generative decoder and the dimensions that lead to the Alzheimer's diagnosis, we generate synthetic dementia patients from MCI patients in the dataset. Experimental results show promising quantitative and qualitative results on one of the most extensive and commonly used Alzheimer's disease neuroimaging datasets in literature.



There are no comments yet.


page 6

page 7


A Graph Gaussian Embedding Method for Predicting Alzheimer's Disease Progression with MEG Brain Networks

Characterizing the subtle changes of functional brain networks associate...

Statistical Disease Progression Modeling in Alzheimer Disease

Background: The characterizing symptom of Alzheimer disease (AD) is cogn...

Predicting Conversion of Mild Cognitive Impairments to Alzheimer's Disease and Exploring Impact of Neuroimaging

Nowadays, a lot of scientific efforts are concentrated on the diagnosis ...

Prediction of Progression to Alzheimer`s disease with Deep InfoMax

Arguably, unsupervised learning plays a crucial role in the majority of ...

Automatic Reminiscence Therapy for Dementia

With people living longer than ever, the number of cases with dementia s...

Identification and classification of TCM syndrome types among patients with vascular mild cognitive impairment using latent tree analysis

Objective: To treat patients with vascular mild cognitive impairment (VM...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.


As the average lifespan increases, neurodegenerative diseases are becoming a common concern threatening millions of lives worldwide. Alzheimer’s disease (AD) is one of the most well-known causes of dementia that affect cognitive functions Hou et al. (2019). According to the World Health Organization, there are approximately 50 million dementia patients in the world Organization (2020). The number of AD patients in the world is expected to exceed 131 million by 2050 Jiang et al. (2018). As the number of patients grows, researchers have been investigating the cause of AD. Fortunately, years of research shed light upon many unknowns. For instance, it is known that age, environment, and medications can trigger AD Hou et al. (2019). On the other hand, there are still many unknowns regarding the root cause and treatment of AD.

Dementia is a condition that impairs memory, language, mood, skills, recognition, judgment, and behavior such that the daily activities are degraded for Disease Control and Prevention (2019). The transition between normal cognitive functions and dementia is called Mild Cognitive Impairment (MCI) Libon et al. (2014)

. Since the effect of the MCI on daily activities may not be noticeable, it is not classified as dementia 

McKhann et al. (2011). While 1-2% of the cognitively normal elderly develop dementia each year, approximately 12% of those with MCI convert to dementia Plassman et al. (2008). While some patients with MCI symptoms have AD in the later stages, some patients may remain in the MCI stage or progress very slowly Minhas et al. (2017). Therefore, the prediction of the conversion from MCI to AD is of great importance.

AD can be diagnosed with an accuracy close to 95% Mucke (2009)

. However, for the prediction of early diagnosis, it is imperative to (i) take a detailed clinical history of the patient and their families, (ii) perform a neurological examination, (iii) perform neuropsychological tests appropriate for the education level to evaluate the cognitive functions of the patient, and (iv) evaluate the brain tissue with neuroimaging methods, such as Magnetic Resonance Imaging (MRI), and Positron Emission Tomography (PET). It is also essential to examine a large-scale patient cohort and simultaneously consider various variables. Artificial intelligence offers tools to simultaneously analyze multiple factors for a large patient cohort. In particular, machine learning and deep learning facilitate early diagnosis with predictive models.

Although deep learning techniques are very successful, they are not easily interpretable. In the healthcare domain, it is crucial to know the underlying reasons behind the behavior of a model. To fully support the clinical research of dementia, predictive techniques should uncover the behavior of the input variables that leads to the transition between the stages of the disease. In this study, we propose a deep learning framework to discover the factors that lead to the conversion from MCI to AD. For this purpose, a latent space manipulation technique is adopted to obtain the significant attributes and decipher their behavior. The contributions of the proposed study are outlined below.

  • A variational auto-encoder (VAE) network is trained to learn a latent representation for the patient data. The latent space of the encoder does not contain any label information.

  • A latent space manipulation technique is applied to the latent representations of the MCI patients. The generative decoder of the VAE outputs new patient data from the manipulated latent representations. To the best of our knowledge, the proposed study is the first attempt to apply latent space manipulation to tabular patient data.

  • The manipulated patient data that are classified as dementia are obtained via a pre-trained classifier. The significant variables that cause the conversion from MCI to Alzheimer’s disease are obtained via analysis of the change in the attributes of the MCI patients and the generated dementia patients from the MCI patients.

  • We investigate correlations between the change in variables that cause the label of a patient flipping from MCI to dementia.

  • We discover relationships between the principal directions in the latent space and the dementia diagnosis.

  • We show that the data of the synthetically generated dementia patients and the real MCI patients, who are diagnosed with dementia in the future time steps, share similar characteristics. This finding is a piece of evidence that the latent space manipulation applied in the proposed study may correspond to a clinically realistic transition between MCI and dementia. As a result, the proposed framework may reveal more information about a patient cohort than a deep classifier.

We conduct experiments with the dataset of The Alzheimer’s Disease Prediction of Longitudinal Evolution, TADPOLE challenge Marinescu et al. (2020). The credibility of the experimental results from a clinical perspective is confirmed with a neurologist.

Figure 1:

Proposed framework. A variational autoencoder and a binary classifier are trained using MCI and dementia patients. Principal components of the latent representations of the patients are computed. We empirically observe the first principal component as the direction of dementia in the latent space. Latent representation of an MCI patient is manipulated by adding the first principal component. Manipulated representations are decoded to obtain the synthetic patients denoted by

. Labels of the synthetic patients are verified using the binary classifier.

Related Work

MCI is defined as a transitional stage between normal aging and dementia Rabin et al. (2009). According to the clinical studies, not all MCI patients convert to AD Chapman et al. (2011). For this reason, detection of the MCI patients who will likely develop AD is of utmost importance for targeted clinical interventions. Therefore, there is a growing literature on the topic. Some studies solely focus on imaging techniques, while others consider the results of cognitive and physiological tests. In this section, we review clinical studies with traditional machine learning models as well as, the studies in the computer science domain which propose novel methods to solve the problem.

Clinical Studies on the Transition Between MCI and AD

Biological and behavioral markers obtained through lab and neuropsychological tests are often used to detect AD Chapman et al. (2011). There has been a quest for finding noninvasive distinctive identifiers and the most informative way of fusing multiple identifiers to diagnose a patient with dementia. For instance, Lu et al. propose to fuse AD biomarkers with an event-based probabilistic framework, named CARE index Lu et al. (2020). The authors state that although screening abnormal amyloid and tau proteins has been essential for accurate diagnosis, it is not always possible Lu et al. (2020). For instance, amyloid-sensitive PET is required to measure the level of amyloid Teipel et al. (2020). Therefore, it is necessary to enable accurate and early diagnosis of AD considering the biomarkers obtained through noninvasive screening. In such cases, it is essential to analyze multiple factors simultaneously. For instance, Sun et al. consider a hierarchical Bayesian model to extract underlying factors from atrophy patterns and cognitive impairments simultaneously Sun et al. (2019).

Mofrad et al. also focus on the abnormalities in brain atrophy due to aging Mofrad et al. (2021). The authors apply an ensemble of predictive models on MRI images to predict the conversion from cognitively normal patient to MCI and from MCI to AD Mofrad et al. (2021). Bae et al.

also focus on the MRI data however they prefer a 3-dimensional Convolutional Neural Network (CNN) to extract features from structural MRI images 

Bae et al. (2021). When more complex models are utilized, an adequate amount of data is required. However, it is often not possible to attain large-scale patient data. Therefore, the authors resort to fine-tuning a pre-trained CNN Bae et al. (2021). Instead of directly utilizing the MRI images, some studies aim to discover new biomarkers from MRI. For instance, Kung et al. propose a biomarker, named ratio of principal curvatures (RPC), that may be an identifier to detect the transition between MCI and AD Kung et al. (2021). Kuang et al.

, on the other hand, considered a drastically different dataset which is collected via a cognitive questionnaire. The authors work with less complex models, such as logistic regression, and Multi-layer Perceptron (MLP). Although the models used in the study may be more interpretable, relying only on the questionnaire dataset is a limitation 

Kuang et al. (2021).

Deep Learning for Neurodegenerative Diseases

In computer science, various healthcare tasks, such as the progression of neurodegenerative diseases Termine et al. (2021); Nguyen et al. (2020), subtyping patients Baytas et al. (2017)

, and early prediction of the conversion from MCI to AD, are usually posed as supervised and unsupervised learning problems with various challenges. Some of these challenges stem from the temporal, heterogeneous, multivariate, multi-view, and non-linear nature of the patient data. Therefore, traditional machine learning problems may not always be competent to tackle them. Consequently, deep learning techniques are preferred in healthcare due to their ability to capture complex interactions between data points.

Deep learning research for early diagnosis of AD comprises of two frequently encountered types of studies. There is a substantial amount of work regarding CNN architectures for feature extraction and segmentation from neuroimaging 

Basheera and Ram (2020); Amini et al. (2021); Wang et al. (2020); Parmar et al. (2020); AbdulAzeem et al. (2021); Xia et al. (2020). These studies utilize effective deep frameworks to predict AD diagnosis. However, they do not address the revelation of significant factors behind the AD diagnosis due to the lack of interpretability. The second line of research is concerned with designing deep frameworks to learn predictive models from clinical, cognitive, neurophysiological test results, and genotype Venugopalan et al. (2021); Bringas et al. (2020); Li et al. (2021); Zhou et al. (2021). In this category, some studies are aiming to extract interpretable features. For instance, Venugopalan et al. extract features from multiple modalities, such as, clinical, genetic, and imaging using a stacked denoising auto-encoder Venugopalan et al. (2021). The authors address the interpretability challenge via a feature masking strategy where one feature is discarded at a time and a decrease in prediction accuracy is observed. According to the experiments, hippocampus, amygdala brain areas, and Rey Auditory Verbal Learning Test are identified as the most significant features for early detection of AD. Although Venugopalan et al. Venugopalan et al. (2021) points to a similar direction as this study, our proposed framework is concerned with deciphering the underlying reasons for the transition between MCI and AD. We model this transition using a latent space manipulation technique.

Latent Space Manipulation

Latent space manipulation is a relatively new concept that is usually applied to Generative Adversarial Networks (GANs). Controlling the latent space enables reshaping the generated output in certain ways. For example, one of the vision studies employs latent space manipulation to generate face images that look more feminine, older, or with more makeup 

Shen et al. (2019). Another study from the healthcare domain utilizes latent space manipulation for high-resolution medical image synthesis using GANs Fetty et al. (2020). Learning to control the latent space can be both during training or after training (by using a pre-trained model) as in this study.

The latent space manipulation can be learned via supervised and unsupervised learning frameworks. In the supervised techniques, an attribute classifier is trained to discover relevant directions within the latent space Goetschalckx et al. (2019)

. Principal Component Analysis (PCA) can also be applied to data points sampled from a latent space to find relevant directions 

Härkönen et al. (2020). DeVries et al.

, on the other hand, propose to apply simple transformations, such as extrapolation, interpolation, and adding Gaussian noise, to the latent representations learned via an auto-encoder 

DeVries and Taylor (2017). A more recent study proposes an automatic, modality-ignorant approach where the latent space is manipulated by extrapolation and interpolation Cheung and Yeung (2021). The studies based on simple transformations aim data augmentation rather than generating very different samples than the input.

In this study, we pursue unsupervised latent space manipulation with PCA. To the best of our knowledge, this is the first attempt to employ a latent space manipulation technique for investigating a healthcare problem using non-vision data.


1:Obtain the latent representations of the patients:
3:Compute the principal component:
5:Manipulate the selected correctly classified MCI patients
7:Reconstruct the manipulated latent representations
Algorithm 1 Latent space manipulation for the MCI patients and synthesis of new patients.

In this section, we present the proposed framework shown in Figure 1. The proposed model consists of a VAE and a binary classifier. Problem definition and the details of the proposed framework are presented in the following sections.

Problem Definition

Predicting the conversion from MCI to AD can be posed as a supervised learning problem. For this purpose, one would need a collection of MCI and AD patients. Thus, a binary classifier can be learned to distinguish the two categories. On the other hand, the goal of the proposed study is to investigate the reasons behind this progression. Therefore, the problem might also be posed as an unsupervised learning problem, where the patient characteristics are learned from the data. A classifier model is still learned to evaluate the outputs of the proposed unsupervised procedure. Throughout the study, we look for the answer to the following questions.

  • Which input attributes do play a significant role in the progression of the MCI to dementia?

  • How do the significant attributes behave during the transition between MCI and dementia?

Model Architecture

The proposed framework require to train a VAE and a binary classifier before executing the latent space manipulation module.

Binary Classifier

A binary classifier is trained to distinguish MCI and AD patients. The patient data might be temporal. However, in this study, we focus on each time step separately. Thus, an MLP network is learned on real MCI and AD patient data. The binary cross-entropy loss given below is used to train the classifier.


where denotes the number of training data points, is the groundtruth label of th sample, is the predicted label of the th sample.

The trained classifier is used to verify whether the label of the synthetic patients who are generated after the latent space manipulation shifts from MCI to AD. The probability of the classifier also provides an insight into the manipulated patient and the amount of the manipulation towards the direction of AD. For instance, the high confidence of the classifier for a synthetic patient, who was labeled as AD, may mean that the condition of the manipulated MCI patient was close to being diagnosed with dementia in the future steps.

Variational Autoencoder

The VAE network aims to learn a latent space for the input samples in an unsupervised way. Unlike the traditional autoencoders, the encoder network outputs a distribution rather than projecting the input to a fixed-sized latent vector. Thus, the VAE prevents overfitting to the specific patients in the training set but captures the distributions of MCI and AD patients. Since the decoder network is trained to reconstruct the input from a sample drawn from a normal distribution, a generative property is injected into it. These properties make the VAE’s latent space more versatile for the latent space manipulation task than the traditional autoencoder. In this study, the encoder and decoder networks of the VAE are comprised of fully connected layers with ReLU activation functions, and they are trained to minimize the following loss function.


where is the input, is the reconstructed input,

denote the mean and variance of the distribution learned by the encoder,

is the Kullback-Leibler divergence, and

controls the effect of KL divergence that measures the distance between two distributions.

Latent Space Manipulation

The latent space manipulation techniques are usually utilized with generative models for vision problems. There is a growing number of studies investigating interpretable directions to manipulate the latent space of a GAN to synthesize images Goetschalckx et al. (2019); Shen et al. (2019); Härkönen et al. (2020). In this study, inspired by the vision domain, we propose to use latent space manipulation to investigate the factors that cause MCI patients to develop dementia. To the best of our knowledge, the latent space manipulation techniques were not applied to tabular patient data before. In the visual domain, the effects of the manipulation can be visually recognized on the synthesized images. On the other hand, in this study, we utilize the binary classifier to understand the effects of the manipulation.

Inspired by the latent space manipulation technique proposed by Härkönen et al. (2020), important latent directions are identified by applying PCA to the latent representations of the patients. Then, the latent representation of a patient is manipulated and the decoder of the VAE is used to synthesize a new patient from the manipulated latent representation as in Equation 3.


where is the latent representation learned by VAE, is one of the principal components, denotes the coefficient that controls the amount of manipulation, and denotes the synthesized patient.

In this study, we aim to understand the underlying factors that cause a patient to be diagnosed with dementia. For this reason, we focus on manipulating the latent representations of MCI patients. For this purpose, first, the reconstructed MCI patient vectors are generated via the learned VAE. Then, we select the MCI patients whose reconstructed samples are correctly identified as MCI by the binary classifier for the latent space manipulation procedure. This procedure aims to select the correctly reconstructed data points so that the effect of manipulation on label shift can be clearly observed. After the latent representations of the selected MCI patients are manipulated using the Equation 3, the predicted labels of the synthesized patients are obtained by the binary classifier to investigate the change that MCI patients undergo. The proposed procedure to synthesize new patients is also summarized in Algorithm 1. Detailed analysis of the proposed approach is presented in the following section.

Variable Dementia - mean (± std) MCI - mean (± std) Missing Value Percentage
Clinical Dementia Rating Scale (SB) 5.99 ± 3.07× 1.62 ± 1.11× %29.65
RAVLT immediate 2.03 ± 0.81 × 3.41 ± 1.12 × %30.69
RAVLT forgetting 4.23 ± 1.93 × 4.66 ± 2.49× %30.89
RAVLT learning 1.66 ± 1.72 × 3.99 ± 2.63 × %30.69
ADAS-Cog11 2.29 ± 1.01 × 0.99 ± 0.51 × %30.06
ADAS-Cog13 3.35 ± 1.13 × 1.61 ± 0.71 × %30.74
Functional Activities Questionnaire 1.69 ± 0.75 × 0.34 ± 0.43 × %29.41
Mini-Mental State Examination 2.12 ± 0.46 × 2.74 ± 0.22 × %29.89
Ventricles 5.32 ± 2.54 × 4.19 ± 2.32 × %41.57
Hippocampus 5.51 ± 1.11 × 6.71 ± 1.12 × %46.61
Whole brain volume 0.95 ± 0.11 × 1.02 ± 0.11 × %39.65
Entorhinal cortical volume 2.69 ± 0.72 × 3.51 ± 0.77 × %49.23
Fusiform cortical volume 1.49 ± 0.26 × 1.74 ± 0.28 × %49.23
Middle temporal cortical volume 1.65 ± 0.32 × 1.95 ± 0.29 × %49.23
Intracranial volume 1.54 ± 0.18 × 1.54 ± 0.16 × %37.58
APOE4 0.88 ± 0.71 × 0.55 ± 0.66 × %0.09
Diagnosis - - %30.11
Table 1: Set of variables with their statistics and percentage of missing values. SB: Sum of boxes, ADAS: Alzheimer’s Disease Assessment Scale, RAVLT: Rey Auditory Verbal Learning Test.


Experiments are conducted with TADPOLE dataset Marinescu et al. (2020), which is retrieved from Alzheimer’s Disease Neuroimaging Initiative (ADNI) Petersen et al. (2010). Google Colab 111 is used for training the proposed framework.

Data Preprocessing

TADPOLE data combines MRI, PET, CSF, diffusion tensor imaging (DTI), cognitive tests, some genetic and demographic information from the ADNI database 

Jack et al. (2015). The dataset consists of visit information of subjects who may be normal control (NC), MCI, or AD patients. Each row represents a patient visit, demonstrating the measurements and test scores as features. On average, each subject has different visits up to years from their first examination. The average number of years between the first and last time points is . The TADPOLE challenge recommends variables including diagnosis, neuropsychological test scores, anatomical features derived from T1 MRI, PET, and CSF markers. However, we selected variables given in Table 1 which have less than of missing values.

Missing data is imputed with forward filling method where the most recent measurement of the patient before the missing value is used for the missing value. If a variable cannot be imputed by the forward filling, the average value of the variable is calculated from the visits which have the same diagnostic label. In the experiments, it is noticed that some measurements are anomalies such that their values are not clinically plausible. Such anomalies are removed from the dataset. PET and MRI measurements, namely hippocampus, whole brain, middle temporal cortical, entorhinal, and fusiform volumes are normalized by the intracranial volume of the particular patient. These measurements do not have standard values and vary according to the patient’s head size. Therefore, they are standardized 

Sargolzaei et al. (2015); Voevodskaya et al. (2014).

Since this study analyzes the conversion from MCI to dementia, we eliminate the patients with only MCI and dementia labels. Thus, we have 1343 data points with dementia label and 4021 data points with MCI label. We use the standard scaling method for normalizing the numerical columns. Compared to min-max scaling, we empirically observe that the standard scaling performs better with the VAE.

Figure 2: Manipulation effects of principal components for a fixed value of . The average probability of the classifier for dementia computed using the manipulated MCI patients is the highest for the first principal component.
Figure 3: Manipulation effect of the first principal component with different values. Horizontal axis represents

values, vertical axis represents the average probability of being dementia of the data points after manipulation and the shaded region denotes the standard deviation.

Figure 4: Correlations of the average changes in variables of the MCI patients who convert to dementia after manipulation with the first principal component with varying .
Figure 5: t-SNE of the manipulated data points with their original versions in the classifier’s latent space, with and . Light blue and dark blue colors represent Dementia and MCI data points before manipulation, red and pink colors represent Dementia and MCI data points after manipulation.

Training Scheme

To train the VAE model, hyperparameter optimization is employed with the 5-fold cross-validation. The encoder is designed as three-layer fully connected network with the dimensions of

for the first, second and third layer, respectively. Another hyperparameter to be tuned is in Equation 2. Higher values provide more structured latent space with weaker reconstruction, and lower values provide better reconstruction with less structured latent space Higgins et al. (2016). The parameter is set to after evaluating the results for the hyperparameter space of .

The binary classifier is designed as a two-layer fully connected network. Hidden dimensionality is determined as and

by 5-fold cross-validation. Batch normalization and dropout with a probability of

are used for regularization. Both the binary classifier and variational autoencoder models are trained with batch size of and validation set split ratio of . Early stopping is used as regularization with epochs. Adam optimizer is used with a learning rate scheduler where the decay rate is and the initial learning rate is set to . During training and data preprocessing, the random seed is fixed. The classification performance on test data is of ROC, of precision and of recall.

Variable Manipulated Data vs. Original Data Future Visit Data vs. Original Data
Clinical Dementia Rating Scale (CDRSB) %27.85 %46.29
RAVLT immediate %-8.01 %-7.31
RAVLT forgetting %-0.56 %-2.15
RAVLT learning %-21.01 %-15.52
ADAS-Cog11 %15.43 %15.49
ADAS-Cog13 %13.23 %13.24
Functional Activities Questionnaire (FAQ) %28.38 %47.88
Mini-Mental State Examination %-4.4 %-6.49
Ventricles %5.79 %6.87
Hippocampus %-6.67 %-4.93
Whole brain volume %-2.25 %-2.39
Entorhinal cortical volume %-6.84 %-4.98
Fusiform cortical volume %-3.77 %-2.68
Middle temporal cortical volume %-3.76 %-3.04
Intracranial volume (ICV) %0.99 %-0.1
APOE4 %13.36 %9.59
Table 2: Average changes between the manipulated and original time steps that are labeled as dementia for a fixed . The manipulation results in similar changes as the real life for the majority of the variables. The manipulation effect falls behind on changing CDRSB and FAQ variables accordingly, and misinterprets the effect of ICV.

Results and Discussion

The latent space manipulation experiments are conducted with varying values of . As given in Equation 3, is the coefficient of the principal components (PCs). We focus only on positive values of since we would like to manipulate the latent representation towards the direction of the PC.

Choice of Principal Components

We extract the top PCs from the patient’s latent representations using PCA. Each PC is added to the latent representations of the MCI patients as in Equation 3. The manipulation effect of each PC is evaluated by the number of MCI patients who are classified as dementia after the manipulation. In Figure 2, for a fixed value of , we plot the average probability of the classifier for dementia computed using manipulated MCI patients against the different principal directions. This empirical evidence indicates that the first PC, which captures the most informative direction, moves the latent representation of an MCI patient towards dementia. For the rest of the experiments, the first PC is considered.

Investigating the Effect of

Next, we investigate the effect of varying values. Figure 3 demonstrates the average of softmax probabilities of being dementia against different values for the first principal component. Different values have a non-linear behavior for converting the MCI patients to dementia. We observe similar behavior when the number of manipulated MCI patients labeled as dementia is investigated for each value. As we show in Figure 3, when the value reaches , we obtain the highest incremental manipulation effect.

Clinical Interpretation

As the manipulation effect increases, values of some variables increase and some of them decrease. We first analyze positive and negative relationships between the change in values of the variables when we increase . For this purpose, we focus on the MCI patients who convert to dementia after the manipulation. Correlation between the average change in variables after the manipulation with the first PC for increasing is shown in Figure 4. In the figure, dark-colored cells indicate a positive correlation while the light color cells indicate an opposite behavior. For instance, the values of ADAS-11, ADAS-13, APOE4, FAQ, ICV, and Ventricles increase together while the whole brain volume behaves opposite to them. Another example could be the positive relationship between MMSE and RAVLT measurements. As we move towards dementia, MMSE and RAVLT measurements decrease together. In Figure 4, we do not show if the variables increase or decrease after manipulation. However, this information can be retrieved from Table 2.

To investigate the individual changes in variables after manipulation, we manipulated the MCI patients, who are diagnosed with dementia in future time steps, using the first PC and of . We compare the average changes between the manipulated and original time steps that are labeled as dementia in Table 2. Future visit information after the manipulated data points of the corresponding patients shows that the patients undergo similar changes to be diagnosed with dementia. The results are clinically plausible for the dataset and the variables used in this study. The reported results in this section imply that we can discover a direction that contains a significant amount of information about AD in the latent space of a deep network.

Visualizing Manipulated Data in Classifier’s Latent Space

Figure 5 visualizes manipulated data points with their original counterparts in the latent space of the binary classifier. Since we apply manipulation only on the MCI labeled patients, Dementia and MCI represent manipulated versions of MCI data points. With this visualization, we can observe the diversity in generated data points. As expected, some of the generated dementia patients are close to the boundary of MCI and dementia regions. However, as desired, they are not collapsed in the middle especially for higher as in Figure 4(b). We infer from this experiment that the proposed approach may be suitable for generating realistic synthetic dementia patients under the low data regime.


Detection of the conversion from MCI to AD is essential for timely treatments and decelerate the disease progression. In this study, we propose a deep framework to investigate the factors that cause this conversion. For this purpose, a VAE network and a binary classifier are trained using the TADPOLE challenge dataset Marinescu et al. (2020). We discover that the first principal component computed from the latent representations of MCI and dementia patient visit data provides the direction towards the conversion to dementia. Thus, we manipulate the latent representations of the MCI patients using the first principal component and obtain dementia patients. The experimental results indicate that it is possible to discover a clinically meaningful direction in the latent space of a deep network. According to the promising results, the proposed framework shows potential to reveal underlying information captured by deep models and use this information to generate realistic synthetic patient data.


  • Y. AbdulAzeem, W. M. Bahgat, and M. Badawy (2021) A cnn based framework for classification of alzheimer’s disease. Neural Computing and Applications, pp. 1–14. Cited by: Deep Learning for Neurodegenerative Diseases.
  • M. Amini, M. Pedram, A. Moradi, and M. Ouchani (2021)

    Diagnosis of alzheimer’s disease severity with fmri images using robust multitask feature extraction method and convolutional neural network (cnn)

    Computational and Mathematical Methods in Medicine 2021. Cited by: Deep Learning for Neurodegenerative Diseases.
  • J. Bae, J. Stocks, A. Heywood, Y. Jung, L. Jenkins, V. Hill, A. Katsaggelos, K. Popuri, H. Rosen, M. F. Beg, et al. (2021) Transfer learning for predicting conversion from mild cognitive impairment to dementia of alzheimer’s type based on a three-dimensional convolutional neural network. Neurobiology of Aging 99, pp. 53–64. Cited by: Clinical Studies on the Transition Between MCI and AD.
  • S. Basheera and M. S. S. Ram (2020) A novel cnn based alzheimer’s disease classification using hybrid enhanced ica segmented gray matter of mri. Computerized Medical Imaging and Graphics 81, pp. 101713. Cited by: Deep Learning for Neurodegenerative Diseases.
  • I. M. Baytas, C. Xiao, X. Zhang, F. Wang, A. K. Jain, and J. Zhou (2017) Patient subtyping via time-aware LSTM networks. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, pp. 65–74. Cited by: Deep Learning for Neurodegenerative Diseases.
  • S. Bringas, S. Salomón, R. Duque, C. Lage, and J. L. Montaña (2020) Alzheimer’s disease stage identification using deep learning models. Journal of Biomedical Informatics 109, pp. 103514. Cited by: Deep Learning for Neurodegenerative Diseases.
  • R. M. Chapman, M. Mapstone, J. W. McCrary, M. N. Gardner, A. Porsteinsson, T. C. Sandoval, M. D. Guillily, E. DeGrush, and L. A. Reilly (2011) Predicting conversion from mild cognitive impairment to alzheimer’s disease using neuropsychological tests and multivariate methods. Journal of clinical and experimental neuropsychology 33 (2), pp. 187–199. Cited by: Clinical Studies on the Transition Between MCI and AD, Related Work.
  • T. Cheung and D. Yeung (2021) {modals}: modality-agnostic automated data augmentation in the latent space. In International Conference on Learning Representations, External Links: Link Cited by: Latent Space Manipulation.
  • T. DeVries and G. W. Taylor (2017) Dataset augmentation in feature space. External Links: 1702.05538 Cited by: Latent Space Manipulation.
  • L. Fetty, M. Bylund, P. Kuess, G. Heilemann, T. Nyholm, D. Georg, and T. Löfstedt (2020) Latent space manipulation for high-resolution medical image synthesis via the stylegan. Zeitschrift für Medizinische Physik 30 (4), pp. 305–314. Cited by: Latent Space Manipulation.
  • C. for Disease Control and Prevention (2019) Alzheimer’s disease and healthy aging. Note: 2021-08-31 Cited by: Introduction.
  • L. Goetschalckx, A. Andonian, A. Oliva, and P. Isola (2019) GANalyze: toward visual definitions of cognitive image properties. CoRR abs/1906.10112. External Links: Link, 1906.10112 Cited by: Latent Space Manipulation, Latent Space Manipulation.
  • E. Härkönen, A. Hertzmann, J. Lehtinen, and S. Paris (2020) GANSpace: discovering interpretable GAN controls. CoRR abs/2004.02546. External Links: Link, 2004.02546 Cited by: Latent Space Manipulation, Latent Space Manipulation, Latent Space Manipulation.
  • I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner (2016) Beta-vae: learning basic visual concepts with a constrained variational framework. Cited by: Training Scheme.
  • Y. Hou, X. Dan, M. Babbar, Y. Wei, S. G. Hasselbalch, D. L. Croteau, and V. A. Bohr (2019) Ageing as a risk factor for neurodegenerative disease. Nature Reviews Neurology 15 (10), pp. 565–581. Cited by: Introduction.
  • C. R. Jack, J. Barnes, M. A. Bernstein, B. J. Borowski, J. Brewer, S. Clegg, A. M. Dale, O. Carmichael, C. Ching, C. DeCarli, R. S. Desikan, C. Fennema-Notestine, A. M. Fjell, E. Fletcher, N. C. Fox, J. Gunter, B. A. Gutman, D. Holland, X. Hua, P. Insel, K. Kantarci, R. J. Killiany, G. Krueger, K. K. Leung, S. Mackin, P. Maillard, I. B. Malone, N. Mattsson, L. McEvoy, M. Modat, S. Mueller, R. Nosheny, S. Ourselin, N. Schuff, M. L. Senjem, A. Simonson, P. M. Thompson, D. Rettmann, P. Vemuri, K. Walhovd, Y. Zhao, S. Zuk, and M. Weiner (2015) Magnetic resonance imaging in alzheimer’s disease neuroimaging initiative 2. Alzheimer’s & Dementia 11 (7), pp. 740–756. External Links: ISSN 1552-5260, Document, Link Cited by: Data Preprocessing.
  • P. Jiang, X. Wang, Q. Li, L. Jin, and S. Li (2018) Correlation-aware sparse and low-rank constrained multi-task learning for longitudinal analysis of alzheimer’s disease. IEEE journal of biomedical and health informatics 23 (4), pp. 1450–1456. Cited by: Introduction.
  • J. Kuang, P. Zhang, T. Cai, Z. Zou, L. Li, N. Wang, and L. Wu (2021)

    Prediction of transition from mild cognitive impairment to alzheimer’s disease based on a logistic regression–artificial neural network–decision tree model

    Geriatrics & Gerontology International 21 (1), pp. 43–47. Cited by: Clinical Studies on the Transition Between MCI and AD.
  • T. Kung, T. Chao, Y. Xie, M. Pai, Y. Kuo, and G. G. C. Lee (2021) Neuroimage biomarker identification of the conversion of mild cognitive impairment to alzheimer’s disease. Frontiers in Neuroscience 15. Cited by: Clinical Studies on the Transition Between MCI and AD.
  • L. Li, Y. Yang, Q. Zhang, J. Wang, J. Jiang, N. Initiative, et al. (2021) Use of deep-learning genomics to discriminate healthy individuals from those with alzheimer’s disease or mild cognitive impairment. Behavioural Neurology 2021. Cited by: Deep Learning for Neurodegenerative Diseases.
  • D.J. Libon, L. Delano-Wood, M.W. Bondi, and R. Au (2014) Mild cognitive impairment. In Encyclopedia of the Neurological Sciences (Second Edition), M. J. Aminoff and R. B. Daroff (Eds.), pp. 72–75. External Links: ISBN 978-0-12-385158-1, Document, Link Cited by: Introduction.
  • X. Lu, J. Chen, H. Shu, Z. Wang, Y. Shi, Y. Yuan, C. Xie, W. Liao, F. Su, Y. Shi, et al. (2020) Predicting conversion to alzheimer’s disease among individual high-risk patients using the characterizing ad risk events index model. CNS neuroscience & therapeutics 26 (7), pp. 720–729. Cited by: Clinical Studies on the Transition Between MCI and AD.
  • R. V. Marinescu, N. P. Oxtoby, A. L. Young, E. E. Bron, A. W. Toga, M. W. Weiner, F. Barkhof, N. C. Fox, A. Eshaghi, T. Toni, M. Salaterski, V. Lunina, M. Ansart, S. Durrleman, P. Lu, S. Iddi, D. Li, W. K. Thompson, M. C. Donohue, A. Nahon, Y. Levy, D. Halbersberg, M. Cohen, H. Liao, T. Li, K. Yu, H. Zhu, J. G. Tamez-Pena, A. Ismail, T. Wood, H. C. Bravo, M. Nguyen, N. Sun, J. Feng, B. T. T. Yeo, G. Chen, K. Qi, S. Chen, D. Qiu, I. Buciuman, A. Kelner, R. Pop, D. Rimocea, M. M. Ghazi, M. Nielsen, S. Ourselin, L. Sorensen, V. Venkatraghavan, K. Liu, C. Rabe, P. Manser, S. M. Hill, J. Howlett, Z. Huang, S. Kiddle, S. Mukherjee, A. Rouanet, B. Taschler, B. D. M. Tom, S. R. White, N. Faux, S. Sedai, J. de Velasco Oriol, E. E. V. Clemente, K. Estrada, L. Aksman, A. Altmann, C. M. Stonnington, Y. Wang, J. Wu, V. Devadas, C. Fourrier, L. L. Raket, A. Sotiras, G. Erus, J. Doshi, C. Davatzikos, J. Vogel, A. Doyle, A. Tam, A. Diaz-Papkovich, E. Jammeh, I. Koval, P. Moore, T. J. Lyons, J. Gallacher, J. Tohka, R. Ciszek, B. Jedynak, K. Pandya, M. Bilgel, W. Engels, J. Cole, P. Golland, S. Klein, and D. C. Alexander (2020) The alzheimer’s disease prediction of longitudinal evolution (tadpole) challenge: results after 1 year follow-up. External Links: 2002.03419 Cited by: Introduction, Experiments, Conclusion.
  • G. M. McKhann, D. S. Knopman, H. Chertkow, B. T. Hyman, C. R. Jack Jr, C. H. Kawas, W. E. Klunk, W. J. Koroshetz, J. J. Manly, R. Mayeux, et al. (2011) The diagnosis of dementia due to alzheimer’s disease: recommendations from the national institute on aging-alzheimer’s association workgroups on diagnostic guidelines for alzheimer’s disease. Alzheimer’s & dementia 7 (3), pp. 263–269. Cited by: Introduction.
  • S. Minhas, A. Khanum, F. Riaz, S. A. Khan, and A. Alvi (2017)

    Predicting progression from mild cognitive impairment to alzheimer’s disease using autoregressive modelling of longitudinal and multimodal biomarkers

    IEEE journal of biomedical and health informatics 22 (3), pp. 818–825. Cited by: Introduction.
  • S. A. Mofrad, A. Lundervold, A. S. Lundervold, A. D. N. Initiative, et al. (2021) A predictive framework based on brain volume trajectories enabling early detection of alzheimer’s disease. Computerized Medical Imaging and Graphics 90, pp. 101910. Cited by: Clinical Studies on the Transition Between MCI and AD.
  • L. Mucke (2009) Alzheimer’s disease. Nature 461 (7266), pp. 895–897. Cited by: Introduction.
  • M. Nguyen, T. He, L. An, D. C. Alexander, J. Feng, and B.T. T. Yeo (2020)

    Predicting alzheimer’s disease progression using deep recurrent neural networks

    NeuroImage 222, pp. 117203. External Links: ISSN 1053-8119, Document, Link Cited by: Deep Learning for Neurodegenerative Diseases.
  • W. H. Organization (2020) Dementia. Note: 2021-08-31 Cited by: Introduction.
  • H. S. Parmar, B. Nutter, R. Long, S. Antani, and S. Mitra (2020) Deep learning of volumetric 3d cnn for fmri in alzheimer’s disease classification. In Medical Imaging 2020: Biomedical Applications in Molecular, Structural, and Functional Imaging, Vol. 11317, pp. 113170C. Cited by: Deep Learning for Neurodegenerative Diseases.
  • R. C. Petersen, P. Aisen, L. A. Beckett, M. Donohue, A. Gamst, D. J. Harvey, C. Jack, W. Jagust, L. Shaw, A. Toga, et al. (2010) Alzheimer’s disease neuroimaging initiative (adni): clinical characterization. Neurology 74 (3), pp. 201–209. Cited by: Experiments.
  • B. L. Plassman, K. M. Langa, G. G. Fisher, S. G. Heeringa, D. R. Weir, M. B. Ofstedal, J. R. Burke, M. D. Hurd, G. G. Potter, W. L. Rodgers, et al. (2008) Prevalence of cognitive impairment without dementia in the united states. Annals of internal medicine 148 (6), pp. 427–434. Cited by: Introduction.
  • L. A. Rabin, N. Paré, A. J. Saykin, M. J. Brown, H. A. Wishart, L. A. Flashman, and R. B. Santulli (2009) Differential memory test sensitivity for diagnosing amnestic mild cognitive impairment and predicting conversion to alzheimer’s disease. Aging, Neuropsychology, and Cognition 16 (3), pp. 357–376. Cited by: Related Work.
  • S. Sargolzaei, A. Sargolzaei, M. Cabrerizo, G. Chen, M. Goryawala, S. Noei, Q. Zhou, R. Duara, W. Barker, and M. Adjouadi (2015)

    A practical guideline for intracranial volume estimation in patients with alzheimer’s disease

    BMC bioinformatics 16 (7), pp. 1–10. Cited by: Data Preprocessing.
  • Y. Shen, J. Gu, X. Tang, and B. Zhou (2019) Interpreting the latent space of gans for semantic face editing. CoRR abs/1907.10786. External Links: Link, 1907.10786 Cited by: Latent Space Manipulation, Latent Space Manipulation.
  • N. Sun, E. C. Mormino, J. Chen, M. R. Sabuncu, and B.T. T. Yeo (2019) Multi-modal latent factor exploration of atrophy, cognitive and tau heterogeneity in alzheimer’s disease. NeuroImage 201, pp. 116043. External Links: ISSN 1053-8119, Document, Link Cited by: Clinical Studies on the Transition Between MCI and AD.
  • S. J. Teipel, M. Dyrba, P. A. Chiesa, F. Sakr, I. Jelistratova, S. Lista, A. Vergallo, P. Lemercier, E. Cavedo, M. O. Habert, et al. (2020) In vivo staging of regional amyloid deposition predicts functional conversion in the preclinical and prodromal phases of alzheimer’s disease. Neurobiology of aging 93, pp. 98–108. Cited by: Clinical Studies on the Transition Between MCI and AD.
  • A. Termine, C. Fabrizio, C. Strafella, V. Caputo, L. Petrosini, C. Caltagirone, E. Giardina, and R. Cascella (2021) Multi-layer picture of neurodegenerative diseases: lessons from the use of big data through artificial intelligence. Journal of Personalized Medicine 11 (4), pp. 280. Cited by: Deep Learning for Neurodegenerative Diseases.
  • J. Venugopalan, L. Tong, H. R. Hassanzadeh, and M. D. Wang (2021) Multimodal deep learning models for early detection of alzheimer’s disease stage. Scientific reports 11 (1), pp. 1–13. Cited by: Deep Learning for Neurodegenerative Diseases.
  • O. Voevodskaya, A. Simmons, R. Nordenskjöld, J. Kullberg, H. Ahlström, L. Lind, L. Wahlund, E. Larsson, E. Westman, A. D. N. Initiative, et al. (2014) The effects of intracranial volume adjustment approaches on multiple regional mri volumes in healthy aging and alzheimer’s disease. Frontiers in aging neuroscience 6, pp. 264. Cited by: Data Preprocessing.
  • N. Wang, M. Chen, and K. P. Subbalakshmi (2020) Explainable cnn-attention networks (c-attention network) for automated detection of alzheimer’s disease. arXiv preprint arXiv:2006.14135. Cited by: Deep Learning for Neurodegenerative Diseases.
  • Z. Xia, G. Yue, Y. Xu, C. Feng, M. Yang, T. Wang, and B. Lei (2020) A novel end-to-end hybrid network for alzheimer’s disease detection using 3d cnn and 3d clstm. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 1–4. Cited by: Deep Learning for Neurodegenerative Diseases.
  • J. Zhou, L. Hu, Y. Jiang, and L. Liu (2021) A correlation analysis between snps and rois of alzheimer’s disease based on deep learning. BioMed Research International 2021. Cited by: Deep Learning for Neurodegenerative Diseases.