Multiscale Progressive Text Prompt Network for Medical Image Segmentation

by   Xianjun Han, et al.

The accurate segmentation of medical images is a crucial step in obtaining reliable morphological statistics. However, training a deep neural network for this task requires a large amount of labeled data to ensure high-accuracy results. To address this issue, we propose using progressive text prompts as prior knowledge to guide the segmentation process. Our model consists of two stages. In the first stage, we perform contrastive learning on natural images to pretrain a powerful prior prompt encoder (PPE). This PPE leverages text prior prompts to generate multimodality features. In the second stage, medical image and text prior prompts are sent into the PPE inherited from the first stage to achieve the downstream medical image segmentation task. A multiscale feature fusion block (MSFF) combines the features from the PPE to produce multiscale multimodality features. These two progressive features not only bridge the semantic gap but also improve prediction accuracy. Finally, an UpAttention block refines the predicted results by merging the image and text features. This design provides a simple and accurate way to leverage multiscale progressive text prior prompts for medical image segmentation. Compared with using only images, our model achieves high-quality results with low data annotation costs. Moreover, our model not only has excellent reliability and validity on medical images but also performs well on natural images. The experimental results on different image datasets demonstrate that our model is effective and robust for image segmentation.


page 1

page 3

page 4

page 8

page 9

page 10

page 13


An Interactive Medical Image Segmentation Framework Using Iterative Refinement

Image segmentation is often performed on medical images for identifying ...

Tailored Multi-Organ Segmentation with Model Adaptation and Ensemble

Multi-organ segmentation, which identifies and separates different organ...

Knowledge-based Fully Convolutional Network and Its Application in Segmentation of Lung CT Images

A variety of deep neural networks have been applied in medical image seg...

Masked Image Modeling Advances 3D Medical Image Analysis

Recently, masked image modeling (MIM) has gained considerable attention ...

Progressive Adversarial Semantic Segmentation

Medical image computing has advanced rapidly with the advent of deep lea...

Leveraging Disease Progression Learning for Medical Image Recognition

Unlike natural images, medical images often have intrinsic characteristi...

Multi-Model Medical Image Segmentation Using Multi-Stage Generative Adversarial Networks

Image segmentation is a challenging problem in medical applications. Med...

Please sign up or login with your details

Forgot password? Click here to reset