Self-Supervised Pretraining for 2D Medical Image Segmentation

09/01/2022
by   András Kalapos, et al.
0

Supervised machine learning provides state-of-the-art solutions to a wide range of computer vision problems. However, the need for copious labelled training data limits the capabilities of these algorithms in scenarios where such input is scarce or expensive. Self-supervised learning offers a way to lower the need for manually annotated data by pretraining models for a specific domain on unlabelled data. In this approach, labelled data are solely required to fine-tune models for downstream tasks. Medical image segmentation is a field where labelling data requires expert knowledge and collecting large labelled datasets is challenging; therefore, self-supervised learning algorithms promise substantial improvements in this field. Despite this, self-supervised learning algorithms are used rarely to pretrain medical image segmentation networks. In this paper, we elaborate and analyse the effectiveness of supervised and self-supervised pretraining approaches on downstream medical image segmentation, focusing on convergence and data efficiency. We find that self-supervised pretraining on natural images and target-domain-specific images leads to the fastest and most stable downstream convergence. In our experiments on the ACDC cardiac segmentation dataset, this pretraining approach achieves 4-5 times faster fine-tuning convergence compared to an ImageNet pretrained model. We also show that this approach requires less than five epochs of pretraining on domain-specific data to achieve such improvement in the downstream convergence time. Finally, we find that, in low-data scenarios, supervised ImageNet pretraining achieves the best accuracy, requiring less than 100 annotated samples to realise close to minimal error.

READ FULL TEXT
research
07/10/2021

Hierarchical Self-Supervised Learning for Medical Image Segmentation Based on Multi-Domain Data Aggregation

A large labeled dataset is a key to the success of supervised deep learn...
research
01/16/2022

Is Contrastive Learning Suitable for Left Ventricular Segmentation in Echocardiographic Images?

Contrastive learning has proven useful in many applications where access...
research
06/29/2023

MIS-FM: 3D Medical Image Segmentation using Foundation Models Pretrained on a Large-Scale Unannotated Dataset

Pretraining with large-scale 3D volumes has a potential for improving th...
research
09/18/2023

Self-supervised TransUNet for Ultrasound regional segmentation of the distal radius in children

Supervised deep learning offers great promise to automate analysis of me...
research
09/01/2020

Quality-aware semi-supervised learning for CMR segmentation

One of the challenges in developing deep learning algorithms for medical...
research
09/07/2022

Prior Knowledge-Guided Attention in Self-Supervised Vision Transformers

Recent trends in self-supervised representation learning have focused on...
research
12/18/2020

Multimodal Transfer Learning-based Approaches for Retinal Vascular Segmentation

In ophthalmology, the study of the retinal microcirculation is a key iss...

Please sign up or login with your details

Forgot password? Click here to reset