Mixture separability loss in a deep convolutional network for image classification

by   Trung Dung Do, et al.

In machine learning, the cost function is crucial because it measures how good or bad a system is. In image classification, well-known networks only consider modifying the network structures and applying cross-entropy loss at the end of the network. However, using only cross-entropy loss causes a network to stop updating weights when all training images are correctly classified. This is the problem of the early saturation. This paper proposes a novel cost function, called mixture separability loss (MSL), which updates the weights of the network even when most of the training images are accurately predicted. MSL consists of between-class and within-class loss. Between-class loss maximizes the differences between inter-class images, whereas within-class loss minimizes the similarities between intra-class images. We designed the proposed loss function to attach to different convolutional layers in the network in order to utilize intermediate feature maps. Experiments show that a network with MSL deepens the learning process and obtains promising results with some public datasets, such as Street View House Number (SVHN), Canadian Institute for Advanced Research (CIFAR), and our self-collected Inha Computer Vision Lab (ICVL) gender dataset.


page 1

page 4

page 6


Negative Log Likelihood Ratio Loss for Deep Neural Network Classification

In deep neural network, the cross-entropy loss function is commonly used...

A New Loss Function for CNN Classifier Based on Pre-defined Evenly-Distributed Class Centroids

With the development of convolutional neural networks (CNNs) in recent y...

What's in a Loss Function for Image Classification?

It is common to use the softmax cross-entropy loss to train neural netwo...

SimLoss: Class Similarities in Cross Entropy

One common loss function in neural network classification tasks is Categ...

σ^2R Loss: a Weighted Loss by Multiplicative Factors using Sigmoidal Functions

In neural networks, the loss function represents the core of the learnin...

Boosting Video Captioning with Dynamic Loss Network

Video captioning is one of the challenging problems at the intersection ...

Redesigning the classification layer by randomizing the class representation vectors

Neural image classification models typically consist of two components. ...