Predicting Brain Degeneration with a Multimodal Siamese Neural Network

11/02/2020
by   Cecilia Ostertag, et al.
0

To study neurodegenerative diseases, longitudinal studies are carried on volunteer patients. During a time span of several months to several years, they go through regular medical visits to acquire data from different modalities, such as biological samples, cognitive tests, structural and functional imaging. These variables are heterogeneous but they all depend on the patient's health condition, meaning that there are possibly unknown relationships between all modalities. Some information may be specific to some modalities, others may be complementary, and others may be redundant. Some data may also be missing. In this work we present a neural network architecture for multimodal learning, able to use imaging and clinical data from two time points to predict the evolution of a neurodegenerative disease, and robust to missing values. Our multimodal network achieves 92.5% accuracy and an AUC score of 0.978 over a test set of 57 subjects. We also show the superiority of the multimodal architecture, for up to 37.5% of missing values in test set subjects' clinical measurements, compared to a model using only the clinical modality.

READ FULL TEXT
research
02/22/2020

Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion

Accurate medical image segmentation commonly requires effective learning...
research
05/25/2023

Incomplete Multimodal Learning for Complex Brain Disorders Prediction

Recent advancements in the acquisition of various brain data sources hav...
research
03/28/2023

Multimodal and multicontrast image fusion via deep generative models

Recently, it has become progressively more evident that classic diagnost...
research
04/06/2023

Longitudinal Multimodal Transformer Integrating Imaging and Latent Clinical Signatures From Routine EHRs for Pulmonary Nodule Classification

The accuracy of predictive models for solitary pulmonary nodule (SPN) di...
research
10/28/2022

M^3Care: Learning with Missing Modalities in Multimodal Healthcare Data

Multimodal electronic health record (EHR) data are widely used in clinic...
research
01/27/2021

G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification

We propose a novel deep neural network architecture to integrate imaging...
research
01/06/2020

Meta-modal Information Flow: A Method for Capturing Multimodal Modular Disconnectivity in Schizophrenia

Objective: Multimodal measurements of the same phenomena provide complem...

Please sign up or login with your details

Forgot password? Click here to reset