DatasetEquity: Are All Samples Created Equal? In The Quest For Equity Within Datasets

by   Shubham Shrivastava, et al.

Data imbalance is a well-known issue in the field of machine learning, attributable to the cost of data collection, the difficulty of labeling, and the geographical distribution of the data. In computer vision, bias in data distribution caused by image appearance remains highly unexplored. Compared to categorical distributions using class labels, image appearance reveals complex relationships between objects beyond what class labels provide. Clustering deep perceptual features extracted from raw pixels gives a richer representation of the data. This paper presents a novel method for addressing data imbalance in machine learning. The method computes sample likelihoods based on image appearance using deep perceptual embeddings and clustering. It then uses these likelihoods to weigh samples differently during training with a proposed Generalized Focal Loss function. This loss can be easily integrated with deep learning algorithms. Experiments validate the method's effectiveness across autonomous driving vision datasets including KITTI and nuScenes. The loss function improves state-of-the-art 3D object detection methods, achieving over 200% AP gains on under-represented classes (Cyclist) in the KITTI dataset. The results demonstrate the method is generalizable, complements existing techniques, and is particularly beneficial for smaller datasets and rare classes. Code is available at:


page 2

page 8

page 11

page 12

page 13

page 14

page 15


Focal Loss in 3D Object Detection

3D object detection is still an open problem in autonomous driving scene...

Resolving Class Imbalance in Object Detection with Weighted Cross Entropy Losses

Object detection is an important task in computer vision which serves a ...

ePose: Let's Make EfficientPose More Generally Applicable

EfficientPose is an impressive 3D object detection model. It has been de...

Balanced Energy Regularization Loss for Out-of-distribution Detection

In the field of out-of-distribution (OOD) detection, a previous method t...

Image to Sphere: Learning Equivariant Features for Efficient Pose Prediction

Predicting the pose of objects from a single image is an important but d...

Homography Loss for Monocular 3D Object Detection

Monocular 3D object detection is an essential task in autonomous driving...

SMOTE-ENC: A novel SMOTE-based method to generate synthetic data for nominal and continuous features

Real world datasets are heavily skewed where some classes are significan...

Please sign up or login with your details

Forgot password? Click here to reset