Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection

by   Chunjong Park, et al.
University of Washington

Unpredictable ML model behavior on unseen data, especially in the health domain, raises serious concerns about its safety as repercussions for mistakes can be fatal. In this paper, we explore the feasibility of using state-of-the-art out-of-distribution detectors for reliable and trustworthy diagnostic predictions. We select publicly available deep learning models relating to various health conditions (e.g., skin cancer, lung sound, and Parkinson's disease) using various input data types (e.g., image, audio, and motion data). We demonstrate that these models show unreasonable predictions on out-of-distribution datasets. We show that Mahalanobis distance- and Gram matrices-based out-of-distribution detection methods are able to detect out-of-distribution data with high accuracy for the health models that operate on different modalities. We then translate the out-of-distribution score into a human interpretable CONFIDENCE SCORE to investigate its effect on the users' interaction with health ML applications. Our user study shows that the confidence score helped the participants only trust the results with a high score to make a medical decision and disregard results with a low score. Through this work, we demonstrate that dataset shift is a critical piece of information for high-stake ML applications, such as medical diagnosis and healthcare, to provide reliable and trustworthy predictions to the users.


Uncertainty-Informed Deep Learning Models Enable High-Confidence Predictions for Digital Histopathology

A model's ability to express its own predictive uncertainty is an essent...

Useful Confidence Measures: Beyond the Max Score

An important component in deploying machine learning (ML) in safety-crit...

DeepLens: Interactive Out-of-distribution Data Detection in NLP Models

Machine Learning (ML) has been widely used in Natural Language Processin...

Two-stage Modeling for Prediction with Confidence

The use of neural networks has been very successful in a wide variety of...

Prediction Confidence from Neighbors

The inability of Machine Learning (ML) models to successfully extrapolat...

Can Machine Learning Be Used to Recognize and Diagnose Coughs?

5G is bringing new use cases to the forefront, one of the most prominent...

BEDS-Bench: Behavior of EHR-models under Distributional Shift–A Benchmark

Machine learning has recently demonstrated impressive progress in predic...

Please sign up or login with your details

Forgot password? Click here to reset