Incremental Learning Meets Transfer Learning: Application to Multi-site Prostate MRI Segmentation

06/03/2022
by   Chenyu You, et al.
0

Many medical datasets have recently been created for medical image segmentation tasks, and it is natural to question whether we can use them to sequentially train a single model that (1) performs better on all these datasets, and (2) generalizes well and transfers better to the unknown target site domain. Prior works have achieved this goal by jointly training one model on multi-site datasets, which achieve competitive performance on average but such methods rely on the assumption about the availability of all training data, thus limiting its effectiveness in practical deployment. In this paper, we propose a novel multi-site segmentation framework called incremental-transfer learning (ITL), which learns a model from multi-site datasets in an end-to-end sequential fashion. Specifically, "incremental" refers to training sequentially constructed datasets, and "transfer" is achieved by leveraging useful information from the linear combination of embedding features on each dataset. In addition, we introduce our ITL framework, where we train the network including a site-agnostic encoder with pre-trained weights and at most two segmentation decoder heads. We also design a novel site-level incremental loss in order to generalize well on the target domain. Second, we show for the first time that leveraging our ITL training scheme is able to alleviate challenging catastrophic forgetting problems in incremental learning. We conduct experiments using five challenging benchmark datasets to validate the effectiveness of our incremental-transfer learning approach. Our approach makes minimal assumptions on computation resources and domain-specific expertise, and hence constitutes a strong starting point in multi-site medical image segmentation.

READ FULL TEXT
research
05/30/2020

Critical Assessment of Transfer Learning for Medical Image Segmentation with Fully Convolutional Neural Networks

Transfer learning is widely used for training machine learning models. H...
research
02/09/2020

MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data

Automated prostate segmentation in MRI is highly demanded for computer-a...
research
03/09/2021

Uncertainty-aware Incremental Learning for Multi-organ Segmentation

Most existing approaches to train a unified multi-organ segmentation mod...
research
03/08/2021

Incremental Learning for Multi-organ Segmentation with Partially Labeled Datasets

There exists a large number of datasets for organ segmentation, which ar...
research
07/24/2019

Mixed-Supervised Dual-Network for Medical Image Segmentation

Deep learning-based medical image segmentation models usually require la...
research
11/09/2022

Domain-incremental Cardiac Image Segmentation with Style-oriented Replay and Domain-sensitive Feature Whitening

Contemporary methods have shown promising results on cardiac image segme...
research
06/01/2018

Learn the new, keep the old: Extending pretrained models with new anatomy and images

Deep learning has been widely accepted as a promising solution for medic...

Please sign up or login with your details

Forgot password? Click here to reset