DeepAI AI Chat
Log In Sign Up

Human uncertainty makes classification more robust

08/19/2019
by   Joshua C. Peterson, et al.
Princeton University
0

The classification performance of deep neural networks has begun to asymptote at near-perfect levels. However, their ability to generalize outside the training set and their robustness to adversarial attacks have not. In this paper, we make progress on this problem by training with full label distributions that reflect human perceptual uncertainty. We first present a new benchmark dataset which we call CIFAR10H, containing a full distribution of human labels for each image of the CIFAR10 test set. We then show that, while contemporary classifiers fail to exhibit human-like uncertainty on their own, explicit training on our dataset closes this gap, supports improved generalization to increasingly out-of-training-distribution test datasets, and confers robustness to adversarial attacks.

READ FULL TEXT
02/04/2021

Adversarial Robustness Study of Convolutional Neural Network for Lumbar Disk Shape Reconstruction from MR images

Machine learning technologies using deep neural networks (DNNs), especia...
11/10/2022

On the Ramifications of Human Label Uncertainty

Humans exhibit disagreement during data labeling. We term this disagreem...
08/18/2022

Shadows Aren't So Dangerous After All: A Fast and Robust Defense Against Shadow-Based Adversarial Attacks

Robust classification is essential in tasks like autonomous vehicle sign...
05/23/2022

Learning to Ignore Adversarial Attacks

Despite the strong performance of current NLP models, they can be brittl...
11/30/2017

Measuring the tendency of CNNs to Learn Surface Statistical Regularities

Deep CNNs are known to exhibit the following peculiarity: on the one han...
10/04/2019

Requirements for Developing Robust Neural Networks

Validation accuracy is a necessary, but not sufficient, measure of a neu...
02/03/2021

Downbeat Tracking with Tempo-Invariant Convolutional Neural Networks

The human ability to track musical downbeats is robust to changes in tem...