Multi-modal learning for predicting the genotype of glioma

03/21/2022
by   Yiran Wei, et al.
0

The isocitrate dehydrogenase (IDH) gene mutation is an essential biomarker for the diagnosis and prognosis of glioma. It is promising to better predict glioma genotype by integrating focal tumor image and geometric features with brain network features derived from MRI. Convolutions neural networks show reasonable performance in predicting IDH mutation, which, however, cannot learn from non-Euclidean data, e.g., geometric and network data. In this study, we propose a multi-modal learning framework using three separate encoders to extract features of focal tumor image, tumor geometrics and global brain networks. To mitigate the limited availability of diffusion MRI, we develop a self-supervised approach to generate brain networks from anatomical multi-sequence MRI. Moreover, to extract tumor-related features from the brain network, we design a hierarchical attention module for the brain network encoder. Further, we design a bi-level multi-modal contrastive loss to align the multi-modal features and tackle the domain gap at the focal tumor and global brain. Finally, we propose a weighted population graph to integrate the multi-modal features for genotype prediction. Experimental results on the testing set show that the proposed model outperforms the baseline deep learning models. The ablation experiments validate the performance of different components of the framework. The visualized interpretation corresponds to clinical knowledge with further validation. In conclusion, the proposed learning framework provides a novel approach for predicting the genotype of glioma.

READ FULL TEXT

page 20

page 21

research
09/04/2021

Predicting isocitrate dehydrogenase mutation status in glioma using structural brain networks and graph neural networks

Glioma is a common malignant brain tumor with distinct survival among pa...
research
09/04/2022

Joint Prediction of Meningioma Grade and Brain Invasion via Task-Aware Contrastive Learning

Preoperative and noninvasive prediction of the meningioma grade is impor...
research
01/14/2022

Collaborative learning of images and geometrics for predicting isocitrate dehydrogenase status of glioma

The isocitrate dehydrogenase (IDH) gene mutation status is an important ...
research
10/29/2020

Brain Tumor Segmentation Network Using Attention-based Fusion and Spatial Relationship Constraint

Delineating the brain tumor from magnetic resonance (MR) images is criti...
research
06/13/2022

MMMNA-Net for Overall Survival Time Prediction of Brain Tumor Patients

Overall survival (OS) time is one of the most important evaluation indic...
research
03/19/2023

PheME: A deep ensemble framework for improving phenotype prediction from multi-modal data

Detailed phenotype information is fundamental to accurate diagnosis and ...
research
05/08/2020

Multi-Phase Cross-modal Learning for Noninvasive Gene Mutation Prediction in Hepatocellular Carcinoma

Hepatocellular carcinoma (HCC) is the most common type of primary liver ...

Please sign up or login with your details

Forgot password? Click here to reset