Transfusion: Understanding Transfer Learning with Applications to Medical Imaging

02/14/2019
by   Jojo Yun, et al.
26

With the increasingly varied applications of deep learning, transfer learning has emerged as a critically important technique. However, the central question of how much feature reuse in transfer is the source of benefit remains unanswered. In this paper, we present an in-depth analysis of the effects of transfer, focusing on medical imaging, which is a particularly intriguing setting. Here, transfer learning is extremely popular, but data differences between pretraining and finetuing are considerable, reiterating the question of what is transferred. With experiments on two large scale medical imaging datasets, and CIFAR-10, we find transfer has almost negligible effects on performance, but significantly helps convergence speed. However, in all of these settings, convergence without transfer can be sped up dramatically by using only mean and variance statistics of the pretrained weights. Visualizing the lower layer filters shows that models trained from random initialization do not learn Gabor filters on medical images. We use CCA (canonical correlation analysis) to study the learned representations of the different models, finding that pretrained models are surprisingly similar to random initialization at higher layers. This similarity is evidenced both through model learning dynamics and a transfusion experiment, which explores the convergence speed using a subset of pretrained weights.

READ FULL TEXT

page 3

page 4

page 11

page 12

page 20

research
01/14/2021

Supervised Transfer Learning at Scale for Medical Imaging

Transfer learning is a standard technique to improve performance on task...
research
03/02/2022

What Makes Transfer Learning Work For Medical Images: Feature Reuse Other Factors

Transfer learning is a standard technique to transfer knowledge from one...
research
02/16/2023

Revisiting Hidden Representations in Transfer Learning for Medical Imaging

While a key component to the success of deep learning is the availabilit...
research
07/01/2023

One Copy Is All You Need: Resource-Efficient Streaming of Medical Imaging Data at Scale

Large-scale medical imaging datasets have accelerated development of art...
research
08/10/2022

Generative Transfer Learning: Covid-19 Classification with a few Chest X-ray Images

Detection of diseases through medical imaging is preferred due to its no...
research
07/22/2020

Rethinking CNN Models for Audio Classification

In this paper, we show that ImageNet-Pretrained standard deep CNN models...

Please sign up or login with your details

Forgot password? Click here to reset