DeepAI AI Chat
Log In Sign Up

Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation

by   Luchen Li, et al.
Imperial College London

Our aim is to establish a framework where reinforcement learning (RL) of optimizing interventions retrospectively allows us a regulatory compliant pathway to prospective clinical testing of the learned policies in a clinical deployment. We focus on infections in intensive care units which are one of the major causes of death and difficult to treat because of the complex and opaque patient dynamics, and the clinically debated, highly-divergent set of intervention policies required by each individual patient, yet intensive care units are naturally data rich. In our work, we build on RL approaches in healthcare ("AI Clinicians"), and learn off-policy continuous dosing policy of pharmaceuticals for sepsis treatment using historical intensive care data under partially observable MDPs (POMDPs). POMPDs capture uncertainty in patient state better by taking in all historical information, yielding an efficient representation, which we investigate through ablations. We compensate for the lack of exploration in our retrospective data by evaluating each encountered state with a best-first tree search. We mitigate state distributional shift by optimizing our policy in the vicinity of the clinicians' compound policy. Crucially, we evaluate our model recommendations using not only conventional policy evaluations but a novel framework that incorporates human experts: a model-agnostic pre-clinical evaluation method to estimate the accuracy and uncertainty of clinician's decisions versus our system recommendations when confronted with the same individual patient history ("shadow mode").


page 1

page 2

page 3

page 4


Deep Reinforcement Learning for Sepsis Treatment

Sepsis is a leading cause of mortality in intensive care units and costs...

A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units

The management of invasive mechanical ventilation, and the regulation of...

Predicting the Need for Blood Transfusion in Intensive Care Units with Reinforcement Learning

As critically ill patients frequently develop anemia or coagulopathy, tr...

Optimizing Sequential Medical Treatments with Auto-Encoding Heuristic Search in POMDPs

Health-related data is noisy and stochastic in implying the true physiol...

An Optimal Policy for Patient Laboratory Tests in Intensive Care Units

Laboratory testing is an integral tool in the management of patient care...

ClinicalVis: Supporting Clinical Task-Focused Design Evaluation

Making decisions about what clinical tasks to prepare for is multi-facto...