Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Space

by   Diane Oyen, et al.

Machine learning classifiers have been demonstrated, both empirically and theoretically, to be robust to label noise under certain conditions – notably the typical assumption is that label noise is independent of the features given the class label. We provide a theoretical framework that generalizes beyond this typical assumption by modeling label noise as a distribution over feature space. We show that both the scale and the shape of the noise distribution influence the posterior likelihood; and the shape of the noise distribution has a stronger impact on classification performance if the noise is concentrated in feature space where the decision boundary can be moved. For the special case of uniform label noise (independent of features and the class label), we show that the Bayes optimal classifier for c classes is robust to label noise until the ratio of noisy samples goes above c-1/c (e.g. 90 which we call the tipping point. However, for the special case of class-dependent label noise (independent of features given the class label), the tipping point can be as low as 50 noise distribution targets decision boundaries (label noise is directly dependent on feature space), classification robustness can drop off even at a small scale of noise. Even when evaluating recent label-noise mitigation methods we see reduced accuracy when label noise is dependent on features. These findings explain why machine learning often handles label noise well if the noise distribution is uniform in feature-space; yet it also points to the difficulty of overcoming label noise when it is concentrated in a region of feature space where a decision boundary can move.


page 1

page 2

page 3

page 4


Label noise detection under the Noise at Random model with ensemble filters

Label noise detection has been widely studied in Machine Learning becaus...

Analysis of Softmax Approximation for Deep Classifiers under Input-Dependent Label Noise

Modelling uncertainty arising from input-dependent label noise is an inc...

Learning with Feature-Dependent Label Noise: A Progressive Approach

Label noise is frequently observed in real-world large-scale datasets. T...

A Second-Order Approach to Learning with Instance-Dependent Label Noise

The presence of label noise often misleads the training of deep neural n...

Attribute noise robust binary classification

We consider the problem of learning linear classifiers when both feature...

Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise

Supervised learning under label noise has seen numerous advances recentl...

Cost Sensitive Learning in the Presence of Symmetric Label Noise

In binary classification framework, we are interested in making cost sen...