Variational Disentanglement for Rare Event Modeling

09/17/2020
by   Zidi Xiu, et al.
21

Combining the increasing availability and abundance of healthcare data and the current advances in machine learning methods have created renewed opportunities to improve clinical decision support systems. However, in healthcare risk prediction applications, the proportion of cases with the condition (label) of interest is often very low relative to the available sample size. Though very prevalent in healthcare, such imbalanced classification settings are also common and challenging in many other scenarios. So motivated, we propose a variational disentanglement approach to semi-parametrically learn from rare events in heavily imbalanced classification problems. Specifically, we leverage the imposed extreme-distribution behavior on a latent space to extract information from low-prevalence events, and develop a robust prediction arm that joins the merits of the generalized additive model and isotonic neural nets. Results on synthetic studies and diverse real-world datasets, including mortality prediction on a COVID-19 cohort, demonstrate that the proposed approach outperforms existing alternatives.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/20/2023

A Comprehensive Survey on Rare Event Prediction

Rare event prediction involves identifying and forecasting events with a...
research
05/24/2022

Deep Reinforcement Learning for Multi-class Imbalanced Training

With the rapid growth of memory and computing power, datasets are becomi...
research
06/01/2020

Logistic Regression for Massive Data with Rare Events

This paper studies binary logistic regression for rare events data, or i...
research
12/01/2021

Output-weighted and relative entropy loss functions for deep learning precursors of extreme events

Many scientific and engineering problems require accurate models of dyna...
research
04/13/2020

Uncertainty estimation for classification and risk prediction in medical settings

In a data-scarce field such as healthcare, where models often deliver pr...
research
03/04/2021

Learning to Predict with Supporting Evidence: Applications to Clinical Risk Prediction

The impact of machine learning models on healthcare will depend on the d...

Please sign up or login with your details

Forgot password? Click here to reset