DeepAI AI Chat
Log In Sign Up

Which Samples Should be Learned First: Easy or Hard?

by   Xiaoling Zhou, et al.

An effective weighting scheme for training samples is essential for learning tasks. Numerous weighting schemes have been proposed. Some schemes take the easy-first mode on samples, whereas some others take the hard-first mode. Naturally, an interesting yet realistic question is raised. Which samples should be learned first given a new learning task, easy or hard? To answer this question, three aspects of research are carried out. First, a high-level unified weighted loss is proposed, providing a more comprehensive view for existing schemes. Theoretical analysis is subsequently conducted and preliminary conclusions are obtained. Second, a flexible weighting scheme is proposed to overcome the defects of existing schemes. The three modes, namely, easy/medium/hard-first, can be flexibly switched in the proposed scheme. Third, a wide range of experiments are conducted to further compare the weighting schemes in different modes. On the basis of these works, reasonable answers are obtained. Factors including prior knowledge and data characteristics determine which samples should be learned first in a learning task.


Understanding Difficulty-based Sample Weighting with a Universal Difficulty Measure

Sample weighting is widely used in deep learning. A large number of weig...

Exploring the Learning Difficulty of Data Theory and Measure

As learning difficulty is crucial for machine learning (e.g., difficulty...

Instance-Level Task Parameters: A Robust Multi-task Weighting Framework

Recent works have shown that deep neural networks benefit from multi-tas...

CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning

Modern deep neural networks can easily overfit to biased training data c...

A Stratified Simulation Scheme for Inference in Bayesian Belief Networks

Simulation schemes for probabilistic inference in Bayesian belief networ...

A Mathematical Foundation for Robust Machine Learning based on Bias-Variance Trade-off

A common assumption in machine learning is that samples are independentl...

A New Weighting Scheme in Weighted Markov Model for Predicting the Probability of Drought Episodes

Drought is a complex stochastic natural hazard caused by prolonged short...