Revisiting Long-tailed Image Classification: Survey and Benchmarks with New Evaluation Metrics

02/03/2023
by   Chaowei Fang, et al.
0

Recently, long-tailed image classification harvests lots of research attention, since the data distribution is long-tailed in many real-world situations. Piles of algorithms are devised to address the data imbalance problem by biasing the training process towards less frequent classes. However, they usually evaluate the performance on a balanced testing set or multiple independent testing sets having distinct distributions with the training data. Considering the testing data may have arbitrary distributions, existing evaluation strategies are unable to reflect the actual classification performance objectively. We set up novel evaluation benchmarks based on a series of testing sets with evolving distributions. A corpus of metrics are designed for measuring the accuracy, robustness, and bounds of algorithms for learning with long-tailed distribution. Based on our benchmarks, we re-evaluate the performance of existing methods on CIFAR10 and CIFAR100 datasets, which is valuable for guiding the selection of data rebalancing techniques. We also revisit existing methods and categorize them into four types including data balancing, feature balancing, loss balancing, and prediction balancing, according the focused procedure during the training pipeline.

READ FULL TEXT
research
11/20/2022

Learning from Long-Tailed Noisy Data with Sample Selection and Balanced Loss

The success of deep learning depends on large-scale and well-curated tra...
research
09/01/2022

Combating Noisy Labels in Long-Tailed Image Classification

Most existing methods that cope with noisy labels usually assume that th...
research
04/09/2023

Propheter: Prophetic Teacher Guided Long-Tailed Distribution Learning

The problem of deep long-tailed learning, a prevalent challenge in the r...
research
12/02/2022

Compound Batch Normalization for Long-tailed Image Classification

Significant progress has been made in learning image classification neur...
research
06/30/2023

Dataset balancing can hurt model performance

Machine learning from training data with a skewed distribution of exampl...
research
08/30/2023

Ten New Benchmarks for Optimization

Benchmarks are used for testing new optimization algorithms and their va...
research
03/10/2023

Long-tailed Classification from a Bayesian-decision-theory Perspective

Long-tailed classification poses a challenge due to its heavy imbalance ...

Please sign up or login with your details

Forgot password? Click here to reset