Suppressing Mislabeled Data via Grouping and Self-Attention

10/29/2020
by   Xiaojiang Peng, et al.
0

Deep networks achieve excellent results on large-scale clean data but degrade significantly when learning from noisy labels. To suppressing the impact of mislabeled data, this paper proposes a conceptually simple yet efficient training block, termed as Attentive Feature Mixup (AFM), which allows paying more attention to clean samples and less to mislabeled ones via sample interactions in small groups. Specifically, this plug-and-play AFM first leverages a group-to-attend module to construct groups and assign attention weights for group-wise samples, and then uses a mixup module with the attention weights to interpolate massive noisy-suppressed samples. The AFM has several appealing benefits for noise-robust deep learning. (i) It does not rely on any assumptions and extra clean subset. (ii) With massive interpolations, the ratio of useless samples is reduced dramatically compared to the original noisy ratio. (iii) It jointly optimizes the interpolation weights with classifiers, suppressing the influence of mislabeled data via low attention weights. (iv) It partially inherits the vicinal risk minimization of mixup to alleviate over-fitting while improves it by sampling fewer feature-target vectors around mislabeled data from the mixup vicinal distribution. Extensive experiments demonstrate that AFM yields state-of-the-art results on two challenging real-world noisy datasets: Food101N and Clothing1M. The code will be available at https://github.com/kaiwang960112/AFM.

READ FULL TEXT
research
03/08/2022

Trustable Co-label Learning from Multiple Noisy Annotators

Supervised deep learning depends on massive accurately annotated example...
research
12/05/2021

Hard Sample Aware Noise Robust Learning for Histopathology Image Classification

Deep learning-based histopathology image classification is a key techniq...
research
08/06/2019

Deep Self-Learning From Noisy Labels

ConvNets achieve good results when training from clean data, but learnin...
research
12/10/2022

Towards Scale Balanced 6-DoF Grasp Detection in Cluttered Scenes

In this paper, we focus on the problem of feature learning in the presen...
research
11/19/2022

Robust AUC Optimization under the Supervision of Clean Data

AUC (area under the ROC curve) optimization algorithms have drawn much a...
research
09/20/2023

Towards Robust Few-shot Point Cloud Semantic Segmentation

Few-shot point cloud semantic segmentation aims to train a model to quic...
research
08/11/2021

Cooperative Learning for Noisy Supervision

Learning with noisy labels has gained the enormous interest in the robus...

Please sign up or login with your details

Forgot password? Click here to reset