Exploiting Class Similarity for Machine Learning with Confidence Labels and Projective Loss Functions

Class labels used for machine learning are relatable to each other, with certain class labels being more similar to each other than others (e.g. images of cats and dogs are more similar to each other than those of cats and cars). Such similarity among classes is often the cause of poor model performance due to the models confusing between them. Current labeling techniques fail to explicitly capture such similarity information. In this paper, we instead exploit the similarity between classes by capturing the similarity information with our novel confidence labels. Confidence labels are probabilistic labels denoting the likelihood of similarity, or confusability, between the classes. Often even after models are trained to differentiate between classes in the feature space, the similar classes' latent space still remains clustered. We view this type of clustering as valuable information and exploit it with our novel projective loss functions. Our projective loss functions are designed to work with confidence labels with an ability to relax the loss penalty for errors that confuse similar classes. We use our approach to train neural networks with noisy labels, as we believe noisy labels are partly a result of confusability arising from class similarity. We show improved performance compared to the use of standard loss functions. We conduct a detailed analysis using the CIFAR-10 dataset and show our proposed methods' applicability to larger datasets, such as ImageNet and Food-101N.


Normalized Loss Functions for Deep Learning with Noisy Labels

Robust loss functions are essential for training accurate deep neural ne...

A Unified DRO View of Multi-class Loss Functions with top-N Consistency

Multi-class classification is one of the most common tasks in machine le...

Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates

Learning with noisy labels is a common problem in supervised learning. E...

Circle Loss: A Unified Perspective of Pair Similarity Optimization

This paper provides a pair similarity optimization viewpoint on deep fea...

Adaptive Verifiable Training Using Pairwise Class Similarity

Verifiable training has shown success in creating neural networks that a...

Input Similarity from the Neural Network Perspective

We first exhibit a multimodal image registration task, for which a neura...

Unimodal Distributions for Ordinal Regression

In many real-world prediction tasks, class labels contain information ab...

Please sign up or login with your details

Forgot password? Click here to reset