Rethinking clinical prediction: Why machine learning must consider year of care and feature aggregation

11/30/2018
by   Bret Nestor, et al.
0

Machine learning for healthcare often trains models on de-identified datasets with randomly-shifted calendar dates, ignoring the fact that data were generated under hospital operation practices that change over time. These changing practices induce definitive changes in observed data which confound evaluations which do not account for dates and limit the generalisability of date-agnostic models. In this work, we establish the magnitude of this problem on MIMIC, a public hospital dataset, and showcase a simple solution. We augment MIMIC with the year in which care was provided and show that a model trained using standard feature representations will significantly degrade in quality over time. We find a deterioration of 0.3 AUC when evaluating mortality prediction on data from 10 years later. We find a similar deterioration of 0.15 AUC for length-of-stay. In contrast, we demonstrate that clinically-oriented aggregates of raw features significantly mitigate future deterioration. Our suggested aggregated representations, when retrained yearly, have prediction quality comparable to year-agnostic models.

READ FULL TEXT
research
08/02/2019

Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks

When training clinical prediction models from electronic health records ...
research
07/22/2019

Design of one-year mortality forecast at hospital admission based: a machine learning approach

Background: Palliative care is referred to a set of programs for patient...
research
02/26/2023

A Multidatabase ExTRaction PipEline (METRE) for Facile Cross Validation in Critical Care Research

Transforming raw EHR data into machine learning model-ready inputs requi...
research
10/28/2019

Added Value of Intraoperative Data for Predicting Postoperative Complications: Development and Validation of a MySurgeryRisk Extension

To test the hypothesis that accuracy, discrimination, and precision in p...
research
10/02/2020

Evaluating Progress on Machine Learning for Longitudinal Electronic Healthcare Data

The Large Scale Visual Recognition Challenge based on the well-known Ima...
research
06/29/2023

Length of Stay prediction for Hospital Management using Domain Adaptation

Inpatient length of stay (LoS) is an important managerial metric which i...

Please sign up or login with your details

Forgot password? Click here to reset