UniSeg: A Prompt-driven Universal Segmentation Model as well as A Strong Representation Learner

by   Yiwen Ye, et al.

The universal model emerges as a promising trend for medical image segmentation, paving up the way to build medical imaging large model (MILM). One popular strategy to build universal models is to encode each task as a one-hot vector and generate dynamic convolutional layers at the end of the decoder to extract the interested target. Although successful, it ignores the correlations among tasks and meanwhile is too late to make the model 'aware' of the ongoing task. To address both issues, we propose a prompt-driven Universal Segmentation model (UniSeg) for multi-task medical image segmentation using diverse modalities and domains. We first devise a learnable universal prompt to describe the correlations among all tasks and then convert this prompt and image features into a task-specific prompt, which is fed to the decoder as a part of its input. Thus, we make the model 'aware' of the ongoing task early and boost the task-specific training of the whole decoder. Our results indicate that the proposed UniSeg outperforms other universal models and single-task models on 11 upstream tasks. Moreover, UniSeg also beats other pre-trained models on two downstream datasets, providing the community with a high-quality pre-trained model for 3D medical image segmentation. Code and model are available at https://github.com/yeerwen/UniSeg.


Training Like a Medical Resident: Universal Medical Image Segmentation via Context Prior Learning

A major enduring focus of clinical workflows is disease analytics and di...

Universal Model for 3D Medical Image Analysis

Deep Learning-based methods recently have achieved remarkable progress i...

STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training

Large-scale models pre-trained on large-scale datasets have profoundly a...

Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation

Parameter Efficient Tuning (PET) has gained attention for reducing the n...

Instructive Feature Enhancement for Dichotomous Medical Image Segmentation

Deep neural networks have been widely applied in dichotomous medical ima...

Ladder Fine-tuning approach for SAM integrating complementary network

Recently, foundation models have been introduced demonstrating various t...

UniverSeg: Universal Medical Image Segmentation

While deep learning models have become the predominant method for medica...

Please sign up or login with your details

Forgot password? Click here to reset