Training Deep Learning models with small datasets

12/14/2019
by   Miguel Romero, et al.
0

The growing use of Machine Learning has produced significant advances in many fields. For image-based tasks, however, the use of deep learning remains challenging in small datasets. In this article, we review, evaluate and compare the current state of the art techniques in training neural networks to elucidate which techniques work best for small datasets. We further propose a path forward for the improvement of model accuracy in medical imaging applications. We observed best results from one cycle training, discriminative learning rates with gradual freezing and parameter modification after transfer learning. We also established that when datasets are small, transfer learning plays an important role beyond parameter initialization by reusing previously learned features. Surprisingly we observed that there is little advantage in using pre-trained networks in images from another part of the body compared to Imagenet. On the contrary, if images from the same part of the body are available then transfer learning can produce a significant improvement in performance with as little as 50 images in the training data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/16/2023

Revisiting Hidden Representations in Transfer Learning for Medical Imaging

While a key component to the success of deep learning is the availabilit...
research
05/25/2018

Using transfer learning to detect galaxy mergers

We investigate the use of deep convolutional neural networks (deep CNNs)...
research
03/02/2022

What Makes Transfer Learning Work For Medical Images: Feature Reuse Other Factors

Transfer learning is a standard technique to transfer knowledge from one...
research
10/06/2022

Data-driven Approaches to Surrogate Machine Learning Model Development

We demonstrate the adaption of three established methods to the field of...
research
08/11/2021

Deep Learning Classification of Lake Zooplankton

Plankton are effective indicators of environmental change and ecosystem ...
research
08/10/2022

Generative Transfer Learning: Covid-19 Classification with a few Chest X-ray Images

Detection of diseases through medical imaging is preferred due to its no...
research
09/15/2020

A Visual Analytics Framework for Explaining and Diagnosing Transfer Learning Processes

Many statistical learning models hold an assumption that the training da...

Please sign up or login with your details

Forgot password? Click here to reset