Evaluating Knowledge Transfer In Neural Network for Medical Images

by   Sina Akbarian, et al.

Deep learning and knowledge transfer techniques have permeated the field of medical imaging and are considered as key approaches for revolutionizing diagnostic imaging practices. However, there are still challenges for the successful integration of deep learning into medical imaging tasks due to a lack of large annotated imaging data. To address this issue, we propose a teacher-student learning framework to transfer knowledge from a carefully pre-trained convolutional neural network (CNN) teacher to a student CNN as a way of improving the diagnostic tasks on a small data regime. In this study, we explore the performance of knowledge transfer in the medical imaging setting through a series of experiments. We investigate the proposed network's performance when the student network is trained on a small dataset (target dataset) as well as when teachers and student's domains are distinct. We also examine the proposed network's behavior on the convergence and regularization of the student network during training. The performances of the CNN models are evaluated on three medical imaging datasets including Diabetic Retinopathy, CheXpert, and ChestX-ray8. Our results indicate that the teacher-student learning framework outperforms transfer learning for small imaging datasets. Particularly, the teacher-student learning framework improves the area under the ROC Curve (AUC) of the CNN model on a small sample of CheXpert (n=5k) by 4 and on ChestX-ray8 (n=5.6k) by 9 also demonstrate a clear advantage to favoring teacher-student learning framework for cross-domain knowledge transfer in the medical imaging setting compared to other knowledge transfer techniques such as transfer learning. We observe that the teacher-student network holds a great promise not only to improve the performance of diagnosis but also to reduce overfitting when the dataset is small.


page 1

page 4

page 5


Supervised Transfer Learning at Scale for Medical Imaging

Transfer learning is a standard technique to improve performance on task...

MED-TEX: Transferring and Explaining Knowledge with Less Data from Pretrained Medical Imaging Models

Deep neural network based image classification methods usually require a...

Network-Agnostic Knowledge Transfer for Medical Image Segmentation

Conventional transfer learning leverages weights of pre-trained networks...

Exploring Knowledge Distillation of a Deep Neural Network for Multi-Script identification

Multi-lingual script identification is a difficult task consisting of di...

ExpandNets: Exploiting Linear Redundancy to Train Small Networks

While very deep networks can achieve great performance, they are ill-sui...

Learning from Higher-Layer Feature Visualizations

Driven by the goal to enable sleep apnea monitoring and machine learning...

Multiple Abnormality Detection for Automatic Medical Image Diagnosis Using Bifurcated Convolutional Neural Network

Automating classification and segmentation process of abnormal regions i...

Please sign up or login with your details

Forgot password? Click here to reset