Radius-margin bounds for deep neural networks

by   Mayank Sharma, et al.
Indian Institute of Technology Delhi

Explaining the unreasonable effectiveness of deep learning has eluded researchers around the globe. Various authors have described multiple metrics to evaluate the capacity of deep architectures. In this paper, we allude to the radius margin bounds described for a support vector machine (SVM) with hinge loss, apply the same to the deep feed-forward architectures and derive the Vapnik-Chervonenkis (VC) bounds which are different from the earlier bounds proposed in terms of number of weights of the network. In doing so, we also relate the effectiveness of techniques like Dropout and Dropconnect in bringing down the capacity of the network. Finally, we describe the effect of maximizing the input as well as the output margin to achieve an input noise-robust deep architecture.


page 1

page 2

page 3

page 4


Margin-Based Regularization and Selective Sampling in Deep Neural Networks

We derive a new margin-based regularization formulation, termed multi-ma...

F-SVM: Combination of Feature Transformation and SVM Learning via Convex Relaxation

The generalization error bound of support vector machine (SVM) depends o...

Effect of Various Regularizers on Model Complexities of Neural Networks in Presence of Input Noise

Deep neural networks are over-parameterized, which implies that the numb...

Robust Large Margin Deep Neural Networks

The generalization error of deep neural networks via their classificatio...

Fast SVM-based Feature Elimination Utilizing Data Radius, Hard-Margin, Soft-Margin

Margin maximization in the hard-margin sense, proposed as feature elimin...

Maximing the Margin in the Input Space

We propose a novel criterion for support vector machine learning: maximi...

Please sign up or login with your details

Forgot password? Click here to reset