Synthetic vs Real: Deep Learning on Controlled Noise

11/21/2019
by   Lu Jiang, et al.
25

Performing controlled experiments on noisy data is essential in thoroughly understanding deep learning across a spectrum of noise levels. Due to the lack of suitable datasets, previous research have only examined deep learning on controlled synthetic noise, and real-world noise has never been systematically studied in a controlled setting. To this end, this paper establishes a benchmark of real-world noisy labels at 10 controlled noise levels. As real-world noise possesses unique properties, to understand the difference, we conduct a large-scale study across a variety of noise levels and types, architectures, methods, and training settings. Our study shows that: (1) Deep Neural Networks (DNNs) generalize much better on real-world noise. (2) DNNs may not learn patterns first on real-world noisy data. (3) When networks are fine-tuned, ImageNet architectures generalize well on noisy data. (4) Real-world noise appears to be less harmful, yet it is more difficult for robust DNN methods to improve. (5) Robust learning methods that work well on synthetic noise may not work as well on real-world noise, and vice versa. We hope our benchmark, as well as our findings, will facilitate deep learning research on noisy data.

READ FULL TEXT

page 5

page 10

page 16

research
05/13/2019

Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels

Noisy labels are ubiquitous in real-world datasets, which poses a challe...
research
06/27/2022

Towards Harnessing Feature Embedding for Robust Learning with Noisy Labels

The memorization effect of deep neural networks (DNNs) plays a pivotal r...
research
08/22/2019

The Learning of Fuzzy Cognitive Maps With Noisy Data: A Rapid and Robust Learning Method With Maximum Entropy

Numerous learning methods for fuzzy cognitive maps (FCMs), such as the H...
research
05/22/2021

Generation and Analysis of Feature-Dependent Pseudo Noise for Training Deep Neural Networks

Training Deep neural networks (DNNs) on noisy labeled datasets is a chal...
research
05/27/2019

Emphasis Regularisation by Gradient Rescaling for Training Deep Neural Networks with Noisy Labels

It is fundamental and challenging to train robust and accurate Deep Neur...
research
02/10/2023

Confidence-based Reliable Learning under Dual Noises

Deep neural networks (DNNs) have achieved remarkable success in a variet...
research
06/01/2023

Comparative Study on the Effects of Noise in ML-Based Anxiety Detection

Wearable health devices are ushering in a new age of continuous and noni...

Please sign up or login with your details

Forgot password? Click here to reset