DeepAI
Log In Sign Up

Learning Fast Sample Re-weighting Without Reward Data

09/07/2021
by   Zizhao Zhang, et al.
1

Training sample re-weighting is an effective approach for tackling data biases such as imbalanced and corrupted labels. Recent methods develop learning-based algorithms to learn sample re-weighting strategies jointly with model training based on the frameworks of reinforcement learning and meta learning. However, depending on additional unbiased reward data is limiting their general applicability. Furthermore, existing learning-based sample re-weighting methods require nested optimizations of models and weighting parameters, which requires expensive second-order computation. This paper addresses these two problems and presents a novel learning-based fast sample re-weighting (FSR) method that does not require additional reward data. The method is based on two key ideas: learning from history to build proxy reward data and feature sharing to reduce the optimization cost. Our experiments show the proposed method achieves competitive results compared to state of the arts on label noise robustness and long-tailed recognition, and does so while achieving significantly improved training efficiency. The source code is publicly available at https://github.com/google-research/google-research/tree/master/ieg.

READ FULL TEXT
12/30/2021

Delving into Sample Loss Curve to Embrace Noisy and Imbalanced Data

Corrupted labels and class imbalance are commonly encountered in practic...
11/22/2022

Dynamic Loss For Robust Learning

Label noise and class imbalance commonly coexist in real-world data. Pre...
10/28/2019

Learning Data Manipulation for Augmentation and Weighting

Manipulating data, such as weighting data examples or augmenting with ne...
02/11/2022

CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning

Modern deep neural networks can easily overfit to biased training data c...
10/17/2020

MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler

Imbalanced learning (IL), i.e., learning unbiased models from class-imba...
08/05/2022

Learning to Re-weight Examples with Optimal Transport for Imbalanced Classification

Imbalanced data pose challenges for deep learning based classification m...
05/27/2019

LAW: Learning to Auto Weight

Example weighting algorithm is an effective solution to the training bia...