DeepAI AI Chat
Log In Sign Up

Leveraging Class Similarity to Improve Deep Neural Network Robustness

by   Pooran Singh Negi, et al.
University of Denver
berkeley college

Traditionally artificial neural networks (ANNs) are trained by minimizing the cross-entropy between a provided groundtruth delta distribution (encoded as one-hot vector) and the ANN's predictive softmax distribution. It seems, however, unacceptable to penalize networks equally for missclassification between classes. Confusing the class "Automobile" with the class "Truck" should be penalized less than confusing the class "Automobile" with the class "Donkey". To avoid such representation issues and learn cleaner classification boundaries in the network, this paper presents a variation of cross-entropy loss which depends not only on the sample class but also on a data-driven prior "class-similarity distribution" across the classes encoded in a matrix form. We explore learning the class-similarity distribution using a datadriven method and then show that by training with our modified similarity-driven loss, we obtain slightly better generalization performance over multiple architectures and datasets as well as improved performance on noisy testing scenarios.


Norm-Scaling for Out-of-Distribution Detection

Out-of-Distribution (OoD) inputs are examples that do not belong to the ...

Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks

Deep neural networks have improved image classification dramatically ove...

Deep Learning on Small Datasets without Pre-Training using Cosine Loss

Two things seem to be indisputable in the contemporary deep learning dis...

Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness

Previous work shows that adversarially robust generalization requires la...

Introducing Graph Smoothness Loss for Training Deep Learning Architectures

We introduce a novel loss function for training deep learning architectu...

Language-Aware Soft Prompting for Vision Language Foundation Models

This paper is on soft prompt learning for Vision & Language (V L) mode...