Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning

01/06/2020
by   Han-Jia Ye, et al.
0

We investigate learning a ConvNet classifier with class-imbalanced data. We found that a ConvNet over-fits significantly to the minor classes that do not have sufficient training instances, even if it is trained using vanilla ERM. We conduct a series of analysis and argue that feature deviation between the training and test instances serves as the main cause. We propose to incorporate class-dependent temperatures (CDT) in learning a ConvNet: CDT forces the minor-class instances to have larger decision values in training, so as to compensate for the effect of feature deviation in testing. We validate our approach on several benchmark datasets and achieve promising results. Our studies further suggest that class-imbalance data affects traditional machine learning and recent deep learning in very different ways. We hope that our insights can inspire new ways of thinking in resolving class-imbalanced deep learning.

READ FULL TEXT
04/05/2021

Procrustean Training for Imbalanced Deep Learning

Neural networks trained with class-imbalanced data are known to perform ...
12/04/2019

Adjusting Decision Boundary for Class Imbalanced Learning

Training of deep neural networks heavily depends on the data distributio...
07/12/2017

Influence of Resampling on Accuracy of Imbalanced Classification

In many real-world binary classification tasks (e.g. detection of certai...
07/17/2018

Pseudo-Feature Generation for Imbalanced Data Analysis in Deep Learning

We generate pseudo-features by multivariate probability distributions ob...
09/03/2020

MixBoost: Synthetic Oversampling with Boosted Mixup for Handling Extreme Imbalance

Training a classification model on a dataset where the instances of one ...
12/03/2020

ReMix: Calibrated Resampling for Class Imbalance in Deep learning

Class imbalance is a problem of significant importance in applied deep l...
06/01/2018

Deep Imbalanced Learning for Face Recognition and Attribute Prediction

Data for face analysis often exhibit highly-skewed class distribution, i...