DeepAI
Log In Sign Up

σ^2R Loss: a Weighted Loss by Multiplicative Factors using Sigmoidal Functions

09/18/2020
by   Riccardo La Grassa, et al.
0

In neural networks, the loss function represents the core of the learning process that leads the optimizer to an approximation of the optimal convergence error. Convolutional neural networks (CNN) use the loss function as a supervisory signal to train a deep model and contribute significantly to achieving the state of the art in some fields of artificial vision. Cross-entropy and Center loss functions are commonly used to increase the discriminating power of learned functions and increase the generalization performance of the model. Center loss minimizes the class intra-class variance and at the same time penalizes the long distance between the deep features inside each class. However, the total error of the center loss will be heavily influenced by the majority of the instances and can lead to a freezing state in terms of intra-class variance. To address this, we introduce a new loss function called sigma squared reduction loss (σ^2R loss), which is regulated by a sigmoid function to inflate/deflate the error per instance and then continue to reduce the intra-class variance. Our loss has clear intuition and geometric interpretation, furthermore, we demonstrate by experiments the effectiveness of our proposal on several benchmark datasets showing the intra-class variance reduction and overcoming the results obtained with center loss and soft nearest neighbour functions.

READ FULL TEXT
10/12/2020

CC-Loss: Channel Correlation Loss For Image Classification

The loss function is a key component in deep learning models. A commonly...
08/30/2019

Handwritten Chinese Character Recognition by Convolutional Neural Network and Similarity Ranking

Convolution Neural Networks (CNN) have recently achieved state-of-the ar...
04/12/2021

Intra-Class Uncertainty Loss Function for Classification

Most classification models can be considered as the process of matching ...
04/30/2021

Center Prediction Loss for Re-identification

The training loss function that enforces certain training sample distrib...
01/21/2022

Nearest Class-Center Simplification through Intermediate Layers

Recent advances in theoretical Deep Learning have introduced geometric p...
09/12/2021

A Decidability-Based Loss Function

Nowadays, deep learning is the standard approach for a wide range of pro...
04/21/2020

AMC-Loss: Angular Margin Contrastive Loss for Improved Explainability in Image Classification

Deep-learning architectures for classification problems involve the cros...