ALASCA: Rethinking Label Smoothing for Deep Learning Under Label Noise

06/15/2022
by   Jongwoo Ko, et al.
0

As label noise, one of the most popular distribution shifts, severely degrades deep neural networks' generalization performance, robust training with noisy labels is becoming an important task in modern deep learning. In this paper, we propose our framework, coined as Adaptive LAbel smoothing on Sub-ClAssifier (ALASCA), that provides a robust feature extractor with theoretical guarantee and negligible additional computation. First, we derive that the label smoothing (LS) incurs implicit Lipschitz regularization (LR). Furthermore, based on these derivations, we apply the adaptive LS (ALS) on sub-classifiers architectures for the practical application of adaptive LR on intermediate layers. We conduct extensive experiments for ALASCA and combine it with previous noise-robust methods on several datasets and show our framework consistently outperforms corresponding baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/05/2020

Does label smoothing mitigate label noise?

Label smoothing is commonly used in training deep learning models, where...
research
02/09/2021

Locally Adaptive Label Smoothing for Predictive Churn

Training modern neural networks is an inherently noisy process that can ...
research
10/23/2020

An Investigation of how Label Smoothing Affects Generalization

It has been hypothesized that label smoothing can reduce overfitting and...
research
05/08/2023

LABO: Towards Learning Optimal Label Regularization via Bi-level Optimization

Regularization techniques are crucial to improving the generalization pe...
research
07/01/2020

Temporal Calibrated Regularization for Robust Noisy Label Learning

Deep neural networks (DNNs) exhibit great success on many tasks with the...
research
05/07/2020

ProSelfLC: Progressive Self Label Correction for Target Revising in Label Noise

In this work, we address robust deep learning under label noise (semi-su...
research
10/26/2019

Model-agnostic Approaches to Handling Noisy Labels When Training Sound Event Classifiers

Label noise is emerging as a pressing issue in sound event classificatio...

Please sign up or login with your details

Forgot password? Click here to reset