Multi-modal Fusion for Diabetes Mellitus and Impaired Glucose Regulation Detection

04/12/2016
by   Jinxing Li, et al.
0

Effective and accurate diagnosis of Diabetes Mellitus (DM), as well as its early stage Impaired Glucose Regulation (IGR), has attracted much attention recently. Traditional Chinese Medicine (TCM) [3], [5] etc. has proved that tongue, face and sublingual diagnosis as a noninvasive method is a reasonable way for disease detection. However, most previous works only focus on a single modality (tongue, face or sublingual) for diagnosis, although different modalities may provide complementary information for the diagnosis of DM and IGR. In this paper, we propose a novel multi-modal classification method to discriminate between DM (or IGR) and healthy controls. Specially, the tongue, facial and sublingual images are first collected by using a non-invasive capture device. The color, texture and geometry features of these three types of images are then extracted, respectively. Finally, our so-called multi-modal similar and specific learning (MMSSL) approach is proposed to combine features of tongue, face and sublingual, which not only exploits the correlation but also extracts individual components among them. Experimental results on a dataset consisting of 192 Healthy, 198 DM and 114 IGR samples (all samples were obtained from Guangdong Provincial Hospital of Traditional Chinese Medicine) substantiate the effectiveness and superiority of our proposed method for the diagnosis of DM and IGR, compared to the case of using a single modality.

READ FULL TEXT

page 1

page 6

page 8

page 9

research
07/05/2022

Is a PET all you need? A multi-modal study for Alzheimer's disease using 3D CNNs

Alzheimer's Disease (AD) is the most common form of dementia and often d...
research
04/19/2023

MAMAF-Net: Motion-Aware and Multi-Attention Fusion Network for Stroke Diagnosis

Stroke is a major cause of mortality and disability worldwide from which...
research
07/31/2023

Multi-modal Graph Neural Network for Early Diagnosis of Alzheimer's Disease from sMRI and PET Scans

In recent years, deep learning models have been applied to neuroimaging ...
research
05/27/2023

A Telecare System for Use in Traditional Persian Medicine

Persian Medicine (PM) uses wrist temperature/humidity and pulse to deter...
research
05/12/2022

Subgroup discovery of Parkinson's Disease by utilizing a multi-modal smart device system

In recent years, sensors from smart consumer devices have shown great di...
research
10/26/2017

Deep Multi-Modal Classification of Intraductal Papillary Mucinous Neoplasms (IPMN) with Canonical Correlation Analysis

Pancreatic cancer has the poorest prognosis among all cancer types. Intr...
research
09/03/2021

A Longitudinal Multi-modal Dataset for Dementia Monitoring and Diagnosis

Dementia is a family of neurogenerative conditions affecting memory and ...

Please sign up or login with your details

Forgot password? Click here to reset