A novel multi-scale loss function for classification problems in machine learning

06/04/2021
by   Leonid Berlyand, et al.
0

We introduce two-scale loss functions for use in various gradient descent algorithms applied to classification problems via deep neural networks. This new method is generic in the sense that it can be applied to a wide range of machine learning architectures, from deep neural networks to support vector machines for example. These two-scale loss functions allow to focus the training onto objects in the training set which are not well classified. This leads to an increase in several measures of performance for appropriately-defined two-scale loss functions with respect to the more classical cross-entropy when tested on traditional deep neural networks on the MNIST, CIFAR10, and CIFAR100 data-sets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/17/2023

Alternate Loss Functions Can Improve the Performance of Artificial Neural Networks

All machine learning algorithms use a loss, cost, utility or reward func...
research
01/31/2018

Optimizing Non-decomposable Measures with Deep Networks

We present a class of algorithms capable of directly training deep neura...
research
06/06/2019

Learning in Gated Neural Networks

Gating is a key feature in modern neural networks including LSTMs, GRUs ...
research
12/12/2022

NFResNet: Multi-scale and U-shaped Networks for Deblurring

Multi-Scale and U-shaped Networks are widely used in various image resto...
research
04/16/2021

Controlled abstention neural networks for identifying skillful predictions for classification problems

The earth system is exceedingly complex and often chaotic in nature, mak...
research
01/20/2021

Component Tree Loss Function: Definition and Optimization

In this article, we propose a method to design loss functions based on c...
research
07/03/2017

Automated Problem Identification: Regression vs Classification via Evolutionary Deep Networks

Regression or classification? This is perhaps the most basic question fa...

Please sign up or login with your details

Forgot password? Click here to reset