Training Like a Medical Resident: Universal Medical Image Segmentation via Context Prior Learning

by   Yunhe Gao, et al.

A major enduring focus of clinical workflows is disease analytics and diagnosis, leading to medical imaging datasets where the modalities and annotations are strongly tied to specific clinical objectives. To date, building task-specific segmentation models is intuitive yet a restrictive approach, lacking insights gained from widespread imaging cohorts. Inspired by the training of medical residents, we explore universal medical image segmentation, whose goal is to learn from diverse medical imaging sources covering a range of clinical targets, body regions, and image modalities. Following this paradigm, we propose Hermes, a context prior learning approach that addresses the challenges related to the heterogeneity on data, modality, and annotations in the proposed universal paradigm. In a collection of seven diverse datasets, we demonstrate the appealing merits of the universal paradigm over the traditional task-specific training paradigm. By leveraging the synergy among various tasks, Hermes shows superior performance and model scalability. Our in-depth investigation on two additional datasets reveals Hermes' strong capabilities for transfer learning, incremental learning, and generalization to different downstream tasks. The code is available:


page 2

page 18


UniSeg: A Prompt-driven Universal Segmentation Model as well as A Strong Representation Learner

The universal model emerges as a promising trend for medical image segme...

UniverSeg: Universal Medical Image Segmentation

While deep learning models have become the predominant method for medica...

Empirical Analysis of a Segmentation Foundation Model in Prostate Imaging

Most state-of-the-art techniques for medical image segmentation rely on ...

Instructive Feature Enhancement for Dichotomous Medical Image Segmentation

Deep neural networks have been widely applied in dichotomous medical ima...

MDViT: Multi-domain Vision Transformer for Small Medical Image Segmentation Datasets

Despite its clinical utility, medical image segmentation (MIS) remains a...

Please sign up or login with your details

Forgot password? Click here to reset