Winning Ticket in Noisy Image Classification

02/23/2021
by   Taehyeon Kim, et al.
1

Modern deep neural networks (DNNs) become frail when the datasets contain noisy (incorrect) class labels. Many robust techniques have emerged via loss adjustment, robust loss function, and clean sample selection to mitigate this issue using the whole dataset. Here, we empirically observe that the dataset which contains only clean instances in original noisy datasets leads to better optima than the original dataset even with fewer data. Based on these results, we state the winning ticket hypothesis: regardless of robust methods, any DNNs reach the best performance when trained on the dataset possessing only clean samples from the original (winning ticket). We propose two simple yet effective strategies to identify winning tickets by looking at the loss landscape and latent features in DNNs. We conduct numerical experiments by collaborating the two proposed methods purifying data and existing robust methods for CIFAR-10 and CIFAR-100. The results support that our framework consistently and remarkably improves performance.

READ FULL TEXT

page 1

page 20

page 21

03/25/2021

Transform consistency for learning with noisy labels

It is crucial to distinguish mislabeled samples for dealing with noisy l...
08/15/2019

Improved Mix-up with KL-Entropy for Learning From Noisy Labels

Despite the deep neural networks (DNN) has achieved excellent performanc...
03/30/2018

Joint Optimization Framework for Learning with Noisy Labels

Deep neural networks (DNNs) trained on large-scale datasets have exhibit...
06/07/2018

Dimensionality-Driven Learning with Noisy Labels

Datasets with significant proportions of noisy (incorrect) class labels ...
01/31/2019

Robust Inference via Generative Classifiers for Handling Noisy Labels

Large-scale datasets may contain significant proportions of noisy (incor...
12/24/2020

Identifying Training Stop Point with Noisy Labeled Data

Training deep neural networks (DNNs) with noisy labels is a challenging ...
07/21/2022

Learning from Data with Noisy Labels Using Temporal Self-Ensemble

There are inevitably many mislabeled data in real-world datasets. Becaus...