Towards dynamic multi-modal phenotyping using chest radiographs and physiological data

11/04/2021
by   Nasir Hayat, et al.
0

The healthcare domain is characterized by heterogeneous data modalities, such as imaging and physiological data. In practice, the variety of medical data assists clinicians in decision-making. However, most of the current state-of-the-art deep learning models solely rely upon carefully curated data of a single modality. In this paper, we propose a dynamic training approach to learn modality-specific data representations and to integrate auxiliary features, instead of solely relying on a single modality. Our preliminary experiments results for a patient phenotyping task using physiological data in MIMIC-IV chest radiographs in the MIMIC- CXR dataset show that our proposed approach achieves the highest area under the receiver operating characteristic curve (AUROC) (0.764 AUROC) compared to the performance of the benchmark method in previous work, which only used physiological data (0.740 AUROC). For a set of five recurring or chronic diseases with periodic acute episodes, including cardiac dysrhythmia, conduction disorders, and congestive heart failure, the AUROC improves from 0.747 to 0.798. This illustrates the benefit of leveraging the chest imaging modality in the phenotyping task and highlights the potential of multi-modal learning in medical applications.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/14/2022

MedFuse: Multi-modal fusion with clinical time-series data and chest X-ray images

Multi-modal fusion approaches aim to integrate information from differen...
research
05/04/2023

Learning Missing Modal Electronic Health Records with Unified Multi-modal Data Embedding and Modality-Aware Attention

Electronic Health Record (EHR) provides abundant information through var...
research
06/21/2022

Toward Unpaired Multi-modal Medical Image Segmentation via Learning Structured Semantic Consistency

Integrating multi-modal data to improve medical image analysis has recei...
research
08/11/2018

SmartEAR: Smartwatch-based Unsupervised Learning for Multi-modal Signal Analysis in Opportunistic Sensing Framework

Wrist-bands such as smartwatches have become an unobtrusive interface fo...
research
12/19/2022

Predicting Ejection Fraction from Chest X-rays Using Computer Vision for Diagnosing Heart Failure

Heart failure remains a major public health challenge with growing costs...
research
10/19/2020

Deep Multi-path Network Integrating Incomplete Biomarker and Chest CT Data for Evaluating Lung Cancer Risk

Clinical data elements (CDEs) (e.g., age, smoking history), blood marker...
research
11/13/2022

Early Diagnosis of Chronic Obstructive Pulmonary Disease from Chest X-Rays using Transfer Learning and Fusion Strategies

Chronic obstructive pulmonary disease (COPD) is one of the most common c...

Please sign up or login with your details

Forgot password? Click here to reset