Introducing One Sided Margin Loss for Solving Classification Problems in Deep Networks

06/02/2022
by   Ali Karimi, et al.
0

This paper introduces a new loss function, OSM (One-Sided Margin), to solve maximum-margin classification problems effectively. Unlike the hinge loss, in OSM the margin is explicitly determined with corresponding hyperparameters and then the classification problem is solved. In experiments, we observe that using OSM loss leads to faster training speeds and better accuracies than binary and categorical cross-entropy in several commonly used deep models for classification and optical character recognition problems. OSM has consistently shown better classification accuracies over cross-entropy and hinge losses for small to large neural networks. it has also led to a more efficient training procedure. We achieved state-of-the-art accuracies for small networks on several benchmark datasets of CIFAR10(98.82%), CIFAR100(91.56%), Flowers(98.04%), Stanford Cars(93.91%) with considerable improvements over other loss functions. Moreover, the accuracies are rather better than cross-entropy and hinge loss for large networks. Therefore, we strongly believe that OSM is a powerful alternative to hinge and cross-entropy losses to train deep neural networks on classification tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/31/2022

Xtreme Margin: A Tunable Loss Function for Binary Classification Problems

Loss functions drive the optimization of machine learning algorithms. Th...
research
06/05/2019

Learning to Rank for Plausible Plausibility

Researchers illustrate improvements in contextual encoding strategies vi...
research
04/26/2022

PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functions

Cross-entropy loss and focal loss are the most common choices when train...
research
12/27/2018

Optimal Margin Distribution Network

Recent research about margin theory has proved that maximizing the minim...
research
02/09/2021

Enhancing Audio Augmentation Methods with Consistency Learning

Data augmentation is an inexpensive way to increase training data divers...
research
07/25/2018

A Surprising Linear Relationship Predicts Test Performance in Deep Networks

Given two networks with the same training loss on a dataset, when would ...
research
07/30/2020

Trade-offs in Top-k Classification Accuracies on Losses for Deep Learning

This paper presents an experimental analysis about trade-offs in top-k c...

Please sign up or login with your details

Forgot password? Click here to reset