Exploiting Uncertainties from Ensemble Learners to Improve Decision-Making in Healthcare AI

07/12/2020
by   Yingshui Tan, et al.
9

Ensemble learning is widely applied in Machine Learning (ML) to improve model performance and to mitigate decision risks. In this approach, predictions from a diverse set of learners are combined to obtain a joint decision. Recently, various methods have been explored in literature for estimating decision uncertainties using ensemble learning; however, determining which metrics are a better fit for certain decision-making applications remains a challenging task. In this paper, we study the following key research question in the selection of uncertainty metrics: when does an uncertainty metric outperforms another? We answer this question via a rigorous analysis of two commonly used uncertainty metrics in ensemble learning, namely ensemble mean and ensemble variance. We show that, under mild assumptions on the ensemble learners, ensemble mean is preferable with respect to ensemble variance as an uncertainty metric for decision making. We empirically validate our assumptions and theoretical results via an extensive case study: the diagnosis of referable diabetic retinopathy.

READ FULL TEXT

page 6

page 16

page 23

page 24

research
11/12/2020

When Does Uncertainty Matter?: Understanding the Impact of Predictive Uncertainty in ML Assisted Decision Making

As machine learning (ML) models are increasingly being employed to assis...
research
03/25/2023

Measuring Classification Decision Certainty and Doubt

Quantitative characterizations and estimations of uncertainty are of fun...
research
07/01/2020

An ensemble learning framework based on group decision making

The classification problem is a significant topic in machine learning wh...
research
12/08/2022

Towards Understanding Fairness and its Composition in Ensemble Machine Learning

Machine Learning (ML) software has been widely adopted in modern society...
research
05/26/2019

Ensemble Decision Systems for General Video Game Playing

Ensemble Decision Systems offer a unique form of decision making that al...
research
06/14/2022

Temporal Multimodal Multivariate Learning

We introduce temporal multimodal multivariate learning, a new family of ...
research
10/01/2021

An Ensemble-based Multi-Criteria Decision Making Method for COVID-19 Cough Classification

The objectives of this research are analysing the performance of the sta...

Please sign up or login with your details

Forgot password? Click here to reset