Class-Distribution-Aware Calibration for Long-Tailed Visual Recognition

09/11/2021
by   Mobarakol Islam, et al.
4

Despite impressive accuracy, deep neural networks are often miscalibrated and tend to overly confident predictions. Recent techniques like temperature scaling (TS) and label smoothing (LS) show effectiveness in obtaining a well-calibrated model by smoothing logits and hard labels with scalar factors, respectively. However, the use of uniform TS or LS factor may not be optimal for calibrating models trained on a long-tailed dataset where the model produces overly confident probabilities for high-frequency classes. In this study, we propose class-distribution-aware TS (CDA-TS) and LS (CDA-LS) by incorporating class frequency information in model calibration in the context of long-tailed distribution. In CDA-TS, the scalar temperature value is replaced with the CDA temperature vector encoded with class frequency to compensate for the over-confidence. Similarly, CDA-LS uses a vector smoothing factor and flattens the hard labels according to their corresponding class distribution. We also integrate CDA optimal temperature vector with distillation loss, which reduces miscalibration in self-distillation (SD). We empirically show that class-distribution-aware TS and LS can accommodate the imbalanced data distribution yielding superior performance in both calibration error and predictive accuracy. We also observe that SD with an extremely imbalanced dataset is less effective in terms of calibration performance. Code is available in https://github.com/mobarakol/Class-Distribution-Aware-TS-LS.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/01/2021

Improving Calibration for Long-Tailed Recognition

Deep neural networks may perform poorly when training datasets are heavi...
research
10/11/2021

Instance-based Label Smoothing For Better Calibrated Classification Networks

Label smoothing is widely used in deep neural networks for multi-class c...
research
07/20/2021

Learning ULMFiT and Self-Distillation with Calibration for Medical Dialogue System

A medical dialogue system is essential for healthcare service as providi...
research
05/31/2023

Perception and Semantic Aware Regularization for Sequential Confidence Calibration

Deep sequence recognition (DSR) models receive increasing attention due ...
research
08/16/2023

Dual-Branch Temperature Scaling Calibration for Long-Tailed Recognition

The calibration for deep neural networks is currently receiving widespre...
research
11/28/2022

Class Adaptive Network Calibration

Recent studies have revealed that, beyond conventional accuracy, calibra...
research
03/28/2023

Dice Semimetric Losses: Optimizing the Dice Score with Soft Labels

The soft Dice loss (SDL) has taken a pivotal role in many automated segm...

Please sign up or login with your details

Forgot password? Click here to reset