Dynamic-Aware Loss for Learning with Label Noise

03/21/2023
by   Xiu-Chuan Li, et al.
0

Label noise poses a serious threat to deep neural networks (DNNs). Employing robust loss function which reconciles fitting ability with robustness is a simple but effective strategy to handle this problem. However, the widely-used static trade-off between these two factors contradicts the dynamic nature of DNNs learning with label noise, leading to inferior performance. Therefore, we propose a dynamics-aware loss (DAL) to solve this problem. Considering that DNNs tend to first learn generalized patterns, then gradually overfit label noise, DAL strengthens the fitting ability initially, then gradually increases the weight of robustness. Moreover, at the later stage, we let DNNs put more emphasis on easy examples which are more likely to be correctly labeled than hard ones and introduce a bootstrapping term to further reduce the negative impact of label noise. Both the detailed theoretical analyses and extensive experimental results demonstrate the superiority of our method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/03/2023

Feature Noise Boosts DNN Generalization under Label Noise

The presence of label noise in the training data has a profound impact o...
research
02/27/2022

Synergistic Network Learning and Label Correction for Noise-robust Image Classification

Large training datasets almost always contain examples with inaccurate o...
research
05/27/2019

Combating Label Noise in Deep Learning Using Abstention

We introduce a novel method to combat label noise when training deep neu...
research
10/11/2020

Is It Time to Redefine the Classification Task for Deep Neural Networks?

Deep neural networks (DNNs) is demonstrated to be vulnerable to the adve...
research
01/14/2021

Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model

The drastic increase of data quantity often brings the severe decrease o...
research
10/26/2022

Deep Learning is Provably Robust to Symmetric Label Noise

Deep neural networks (DNNs) are capable of perfectly fitting the trainin...
research
07/17/2022

A Study of Deep CNN Model with Labeling Noise Based on Granular-ball Computing

In supervised learning, the presence of noise can have a significant imp...

Please sign up or login with your details

Forgot password? Click here to reset