Optimal Margin Distribution Network

12/27/2018
by   Shen-Huan Lv, et al.
0

Recent research about margin theory has proved that maximizing the minimum margin like support vector machines does not necessarily lead to better performance, and instead, it is crucial to optimize the margin distribution. In the meantime, margin theory has been used to explain the empirical success of deep network in recent studies. In this paper, we present mdNet (the Optimal Margin Distribution Network), a network which embeds a loss function in regard to the optimal margin distribution. We give a theoretical analysis of our method using the PAC-Bayesian framework, which confirms the significance of the margin distribution for classification within the framework of deep networks. In addition, empirical results show that the mdNet model always outperforms the baseline cross-entropy loss model consistently across different regularization situations. And our mdNet model also outperforms the cross-entropy loss (Xent), hinge loss and soft hinge loss model in generalization task through limited training data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/28/2018

Predicting the Generalization Gap in Deep Networks with Margin Distributions

As shown in recent research, deep neural networks can perfectly fit rand...
research
06/02/2022

Introducing One Sided Margin Loss for Solving Classification Problems in Deep Networks

This paper introduces a new loss function, OSM (One-Sided Margin), to so...
research
01/16/2013

An Uncertainty Framework for Classification

We define a generalized likelihood function based on uncertainty measure...
research
07/15/2022

Support Vector Machines with the Hard-Margin Loss: Optimal Training via Combinatorial Benders' Cuts

The classical hinge-loss support vector machines (SVMs) model is sensiti...
research
08/24/2023

Don't blame Dataset Shift! Shortcut Learning due to Gradients and Cross Entropy

Common explanations for shortcut learning assume that the shortcut impro...
research
04/18/2015

On the consistency of Multithreshold Entropy Linear Classifier

Multithreshold Entropy Linear Classifier (MELC) is a recent classifier i...
research
05/08/2023

Scalable Optimal Margin Distribution Machine

Optimal margin Distribution Machine (ODM) is a newly proposed statistica...

Please sign up or login with your details

Forgot password? Click here to reset