Does deep learning model calibration improve performance in class-imbalanced medical image classification?

by   Sivaramakrishnan Rajaraman, et al.

In medical image classification tasks, it is common to find that the number of normal samples far exceeds the number of abnormal samples. In such class-imbalanced situations, reliable training of deep neural networks continues to be a major challenge. Under these circumstances, the predicted class confidence may be biased toward the majority class. Calibration has been suggested to alleviate some of these effects. However, there is insufficient analysis explaining when and whether calibrating a model would be beneficial in improving performance. In this study, we perform a systematic analysis of the effect of model calibration on its performance on two medical image modalities, namely, chest X-rays (CXRs) and fundus images, using various deep learning classifier backbones. For this, we study the following variations: (i) the degree of imbalances in the dataset used for training; (ii) calibration methods; and, (iii) two classification thresholds, namely, default decision threshold of 0.5, and optimal threshold from precision-recall (PR) curves. Our results indicate that at the default operating threshold of 0.5, the performance achieved through calibration is significantly superior (p < 0.05) to an uncalibrated model. However, at the PR-guided threshold, these gains were not significantly different (p > 0.05). This finding holds for both image modalities and at varying degrees of imbalance.



There are no comments yet.


page 4

page 6

page 7

page 11

page 15

page 16

page 22

page 23


Multi-loss ensemble deep learning for chest X-ray classification

Class imbalance is common in medical image classification tasks, where t...

Medical Knowledge-Guided Deep Learning for Imbalanced Medical Image Classification

Deep learning models have gained remarkable performance on a variety of ...

Learning multiple non-mutually-exclusive tasks for improved classification of inherently ordered labels

Medical image classification involves thresholding of labels that repres...

Semi-supervised learning for medical image classification using imbalanced training data

Medical image classification is often challenging for two reasons: a lac...

Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classification

Recent works have shown that deep neural networks can achieve super-huma...

Understanding the impact of class imbalance on the performance of chest x-ray image classifiers

This work aims to understand the impact of class imbalance on the perfor...

Minority Class Oversampling for Tabular Data with Deep Generative Models

In practice, data scientists are often confronted with imbalanced data. ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.