Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning

by   Han-Jia Ye, et al.

We investigate learning a ConvNet classifier with class-imbalanced data. We found that a ConvNet over-fits significantly to the minor classes that do not have sufficient training instances, even if it is trained using vanilla ERM. We conduct a series of analysis and argue that feature deviation between the training and test instances serves as the main cause. We propose to incorporate class-dependent temperatures (CDT) in learning a ConvNet: CDT forces the minor-class instances to have larger decision values in training, so as to compensate for the effect of feature deviation in testing. We validate our approach on several benchmark datasets and achieve promising results. Our studies further suggest that class-imbalance data affects traditional machine learning and recent deep learning in very different ways. We hope that our insights can inspire new ways of thinking in resolving class-imbalanced deep learning.


Procrustean Training for Imbalanced Deep Learning

Neural networks trained with class-imbalanced data are known to perform ...

Adjusting Decision Boundary for Class Imbalanced Learning

Training of deep neural networks heavily depends on the data distributio...

Influence of Resampling on Accuracy of Imbalanced Classification

In many real-world binary classification tasks (e.g. detection of certai...

Pseudo-Feature Generation for Imbalanced Data Analysis in Deep Learning

We generate pseudo-features by multivariate probability distributions ob...

MixBoost: Synthetic Oversampling with Boosted Mixup for Handling Extreme Imbalance

Training a classification model on a dataset where the instances of one ...

ReMix: Calibrated Resampling for Class Imbalance in Deep learning

Class imbalance is a problem of significant importance in applied deep l...

Deep Imbalanced Learning for Face Recognition and Attribute Prediction

Data for face analysis often exhibit highly-skewed class distribution, i...