Improving image classifiers for small datasets by learning rate adaptations

03/26/2019
by   Sourav Mishra, et al.
0

Our paper introduces an efficient combination of established techniques to improve classifier performance, in terms of accuracy and training time. We achieve two-fold to ten-fold speedup in nearing state of the art accuracy, over different model architectures, by dynamically tuning the learning rate. We find it especially beneficial in the case of a small dataset, where reliability of machine reasoning is lower. We validate our approach by comparing our method versus vanilla training on CIFAR-10. We also demonstrate its practical viability by implementing on an unbalanced corpus of diagnostic images.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/10/2019

Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks

Stochastic gradient descent with a large initial learning rate is a wide...
research
10/25/2019

A Simple Dynamic Learning Rate Tuning Algorithm For Automated Training of DNNs

Training neural networks on image datasets generally require extensive e...
research
07/08/2020

AutoLR: An Evolutionary Approach to Learning Rate Policies

The choice of a proper learning rate is paramount for good Artificial Ne...
research
03/20/2020

Event-Based Control for Online Training of Neural Networks

Convolutional Neural Network (CNN) has become the most used method for i...
research
08/21/2023

We Don't Need No Adam, All We Need Is EVE: On The Variance of Dual Learning Rate And Beyond

In the rapidly advancing field of deep learning, optimising deep neural ...
research
11/18/2019

Feedback Control for Online Training of Neural Networks

Convolutional neural networks (CNNs) are commonly used for image classif...
research
10/24/2022

Selecting and Composing Learning Rate Policies for Deep Neural Networks

The choice of learning rate (LR) functions and policies has evolved from...

Please sign up or login with your details

Forgot password? Click here to reset