DeepAI
Log In Sign Up

SimLoss: Class Similarities in Cross Entropy

03/06/2020
by   Konstantin Kobs, et al.
0

One common loss function in neural network classification tasks is Categorical Cross Entropy (CCE), which punishes all misclassifications equally. However, classes often have an inherent structure. For instance, classifying an image of a rose as "violet" is better than as "truck". We introduce SimLoss, a drop-in replacement for CCE that incorporates class similarities along with two techniques to construct such matrices from task-specific knowledge. We test SimLoss on Age Estimation and Image Classification and find that it brings significant improvements over CCE on several metrics. SimLoss therefore allows for explicit modeling of background knowledge by simply exchanging the loss function, while keeping the neural network architecture the same. Code and additional resources can be found at https://github.com/konstantinkobs/SimLoss.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/27/2018

Negative Log Likelihood Ratio Loss for Deep Neural Network Classification

In deep neural network, the cross-entropy loss function is commonly used...
07/31/2019

Competing Ratio Loss for Discriminative Multi-class Image Classification

The development of deep convolutional neural network architecture is cri...
04/17/2019

Aggregation Cross-Entropy for Sequence Recognition

In this paper, we propose a novel method, aggregation cross-entropy (ACE...
11/16/2020

Redesigning the classification layer by randomizing the class representation vectors

Neural image classification models typically consist of two components. ...
07/27/2019

Forced Spatial Attention for Driver Foot Activity Classification

This paper provides a simple solution for reliably solving image classif...
12/19/2019

Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks

Deep neural networks have improved image classification dramatically ove...
06/16/2019

Mixture separability loss in a deep convolutional network for image classification

In machine learning, the cost function is crucial because it measures ho...

Code Repositories

SimLoss

A PyTorch implementation of our proposed loss function from the paper "SimLoss: Class Similarities in Cross Entropy"


view repo