Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions

by   Abhijit Guha Roy, et al.

We develop and rigorously evaluate a deep learning based system that can accurately classify skin conditions while detecting rare conditions for which there is not enough data available for training a confident classifier. We frame this task as an out-of-distribution (OOD) detection problem. Our novel approach, hierarchical outlier detection (HOD) assigns multiple abstention classes for each training outlier class and jointly performs a coarse classification of inliers vs. outliers, along with fine-grained classification of the individual classes. We demonstrate the effectiveness of the HOD loss in conjunction with modern representation learning approaches (BiT, SimCLR, MICLe) and explore different ensembling strategies for further improving the results. We perform an extensive subgroup analysis over conditions of varying risk levels and different skin types to investigate how the OOD detection performance changes over each subgroup and demonstrate the gains of our framework in comparison to baselines. Finally, we introduce a cost metric to approximate downstream clinical impact. We use this cost metric to compare the proposed method against a baseline system, thereby making a stronger case for the overall system effectiveness in a real-world deployment scenario.


Out-of-Distribution Detection for Long-tailed and Fine-grained Skin Lesion Images

Recent years have witnessed a rapid development of automated methods for...

Fine-grained Out-of-Distribution Detection with Mixup Outlier Exposure

Enabling out-of-distribution (OOD) detection for DNNs is critical for th...

XGBOD: Improving Supervised Outlier Detection with Unsupervised Representation Learning

A new semi-supervised ensemble algorithm called XGBOD (Extreme Gradient ...

Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier

Long-tail recognition tackles the natural non-uniformly distributed data...

Skin cancer detection based on deep learning and entropy to detect outlier samples

We describe our methods to address both tasks of the ISIC 2019 challenge...

Fine-grain Inference on Out-of-Distribution Data with Hierarchical Classification

Machine learning methods must be trusted to make appropriate decisions i...

Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation of "Outlier" Detectors

In the real world, a learning system could receive an input that looks n...

Please sign up or login with your details

Forgot password? Click here to reset