RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR

by   Yuyin Zhou, et al.

Despite the routine use of electronic health record (EHR) data by radiologists to contextualize clinical history and inform image interpretation, the majority of deep learning architectures for medical imaging are unimodal, i.e., they only learn features from pixel-level information. Recent research revealing how race can be recovered from pixel data alone highlights the potential for serious biases in models which fail to account for demographics and other key patient attributes. Yet the lack of imaging datasets which capture clinical context, inclusive of demographics and longitudinal medical history, has left multimodal medical imaging underexplored. To better assess these challenges, we present RadFusion, a multimodal, benchmark dataset of 1794 patients with corresponding EHR data and high-resolution computed tomography (CT) scans labeled for pulmonary embolism. We evaluate several representative multimodal fusion models and benchmark their fairness properties across protected subgroups, e.g., gender, race/ethnicity, age. Our results suggest that integrating imaging and EHR data can improve classification performance and robustness without introducing large disparities in the true positive rate between population groups.


page 1

page 2

page 3

page 4


Longitudinal Multimodal Transformer Integrating Imaging and Latent Clinical Signatures From Routine EHRs for Pulmonary Nodule Classification

The accuracy of predictive models for solitary pulmonary nodule (SPN) di...

An Ensemble Approach for Patient Prognosis of Head and Neck Tumor Using Multimodal Data

Accurate prognosis of a tumor can help doctors provide a proper course o...

X-ray Recognition: Patient identification from X-rays using a contrastive objective

Recent research demonstrates that deep learning models are capable of pr...

Bias and Fairness on Multimodal Emotion Detection Algorithms

Numerous studies have shown that machine learning algorithms can latch o...

RIDDLE: Race and ethnicity Imputation from Disease history with Deep LEarning

Anonymized electronic medical records are an increasingly popular source...

Are demographically invariant models and representations in medical imaging fair?

Medical imaging models have been shown to encode information about patie...

The dynamics of the stomatognathic system from 4D multimodal data

The purpose of this chapter is to discuss methods of acquisition, visual...

Please sign up or login with your details

Forgot password? Click here to reset