DeepAI AI Chat
Log In Sign Up

Statistical Measures For Defining Curriculum Scoring Function

by   Vinu Sankar Sadasivan, et al.

Curriculum learning is a training strategy that sorts the training examples by some measure of their difficulty and gradually exposes them to the learner to improve the network performance. In this work, we propose two novel curriculum learning algorithms, and empirically show their improvements in performance with convolutional and fully-connected neural networks on multiple real image datasets. Motivated by our insights from implicit curriculum ordering, we introduce a simple curriculum learning strategy that uses statistical measures such as standard deviation and entropy values to score the difficulty of data points for real image classification tasks. We also propose and study the performance of a dynamic curriculum learning algorithm. Our dynamic curriculum algorithm tries to reduce the distance between the network weight and an optimal weight at any training step by greedily sampling examples with gradients that are directed towards the optimal weight. Further, we also use our algorithms to discuss why curriculum learning is helpful.


page 1

page 3

page 4

page 5

page 8


Dynamic Data Selection for Curriculum Learning via Ability Estimation

Curriculum learning methods typically rely on heuristics to estimate the...

Spatial Transformer Networks for Curriculum Learning

Curriculum learning is a bio-inspired training technique that is widely ...

LeRaC: Learning Rate Curriculum

Most curriculum learning methods require an approach to sort the data sa...

When Do Curricula Work?

Inspired by human learning, researchers have proposed ordering examples ...

Label-similarity Curriculum Learning

Curriculum learning can improve neural network training by guiding the o...

Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs

Experimental results have shown that curriculum learning, i.e., presenti...

Learn like a Pathologist: Curriculum Learning by Annotator Agreement for Histopathology Image Classification

Applying curriculum learning requires both a range of difficulty in data...