Masked Image Modeling Advances 3D Medical Image Analysis

by   Zekai Chen, et al.

Recently, masked image modeling (MIM) has gained considerable attention due to its capacity to learn from vast amounts of unlabeled data and has been demonstrated to be effective on a wide variety of vision tasks involving natural images. Meanwhile, the potential of self-supervised learning in modeling 3D medical images is anticipated to be immense due to the high quantities of unlabeled images, and the expense and difficulty of quality labels. However, MIM's applicability to medical images remains uncertain. In this paper, we demonstrate that masked image modeling approaches can also advance 3D medical images analysis in addition to natural images. We study how masked image modeling strategies leverage performance from the viewpoints of 3D medical image segmentation as a representative downstream task: i) when compared to naive contrastive learning, masked image modeling approaches accelerate the convergence of supervised training even faster (1.40×) and ultimately produce a higher dice score; ii) predicting raw voxel values with a high masking ratio and a relatively smaller patch size is non-trivial self-supervised pretext-task for medical images modeling; iii) a lightweight decoder or projection head design for reconstruction is powerful for masked image modeling on 3D medical images which speeds up training and reduce cost; iv) finally, we also investigate the effectiveness of MIM methods under different practical scenarios where different image resolutions and labeled data ratios are applied.


page 2

page 3

page 12


A Review of Predictive and Contrastive Self-supervised Learning for Medical Images

Over the last decade, supervised deep learning on manually annotated big...

Quantitative Imaging Principles Improves Medical Image Learning

Fundamental differences between natural and medical images have recently...

MixCL: Pixel label matters to contrastive learning

Contrastive learning and self-supervised techniques have gained prevalen...

AMLP:Adaptive Masking Lesion Patches for Self-supervised Medical Image Segmentation

Self-supervised masked image modeling has shown promising results on nat...

ViT-AE++: Improving Vision Transformer Autoencoder for Self-supervised Medical Image Representations

Self-supervised learning has attracted increasing attention as it learns...

Multiscale Progressive Text Prompt Network for Medical Image Segmentation

The accurate segmentation of medical images is a crucial step in obtaini...

Unified 2D and 3D Pre-training for Medical Image classification and Segmentation

Self-supervised learning (SSL) opens up huge opportunities for better ut...

Please sign up or login with your details

Forgot password? Click here to reset