Dimensionality-Driven Learning with Noisy Labels

06/07/2018
by   Xingjun Ma, et al.
0

Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such datasets, by investigating the dimensionality of the deep representation subspace of training samples. We show that from a dimensionality perspective, DNNs exhibit quite distinctive learning styles when trained with clean labels versus when trained with a proportion of noisy labels. Based on this finding, we develop a new dimensionality-driven learning strategy, which monitors the dimensionality of subspaces during training and adapts the loss function accordingly. We empirically demonstrate that our approach is highly tolerant to significant proportions of noisy labels, and can effectively learn low-dimensional local subspaces that capture the data distribution.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/30/2018

Joint Optimization Framework for Learning with Noisy Labels

Deep neural networks (DNNs) trained on large-scale datasets have exhibit...
research
08/31/2019

Learning with Noisy Labels for Sentence-level Sentiment Classification

Deep neural networks (DNNs) can fit (or even over-fit) the training data...
research
02/23/2021

Winning Ticket in Noisy Image Classification

Modern deep neural networks (DNNs) become frail when the datasets contai...
research
06/13/2019

A Meta Approach to Defend Noisy Labels by the Manifold Regularizer PSDR

Noisy labels are ubiquitous in real-world datasets, which poses a challe...
research
01/31/2019

Robust Inference via Generative Classifiers for Handling Noisy Labels

Large-scale datasets may contain significant proportions of noisy (incor...
research
07/11/2023

Memorization Through the Lens of Curvature of Loss Function Around Samples

Neural networks are overparametrized and easily overfit the datasets the...
research
09/17/2020

Noisy Concurrent Training for Efficient Learning under Label Noise

Deep neural networks (DNNs) fail to learn effectively under label noise ...

Please sign up or login with your details

Forgot password? Click here to reset