DeepAI AI Chat
Log In Sign Up

TransNetR: Transformer-based Residual Network for Polyp Segmentation with Multi-Center Out-of-Distribution Testing

by   Debesh Jha, et al.
iit guwahati
Northwestern University

Colonoscopy is considered the most effective screening test to detect colorectal cancer (CRC) and its precursor lesions, i.e., polyps. However, the procedure experiences high miss rates due to polyp heterogeneity and inter-observer dependency. Hence, several deep learning powered systems have been proposed considering the criticality of polyp detection and segmentation in clinical practices. Despite achieving improved outcomes, the existing automated approaches are inefficient in attaining real-time processing speed. Moreover, they suffer from a significant performance drop when evaluated on inter-patient data, especially those collected from different centers. Therefore, we intend to develop a novel real-time deep learning based architecture, Transformer based Residual network (TransNetR), for colon polyp segmentation and evaluate its diagnostic performance. The proposed architecture, TransNetR, is an encoder-decoder network that consists of a pre-trained ResNet50 as the encoder, three decoder blocks, and an upsampling layer at the end of the network. TransNetR obtains a high dice coefficient of 0.8706 and a mean Intersection over union of 0.8016 and retains a real-time processing speed of 54.60 on the Kvasir-SEG dataset. Apart from this, the major contribution of the work lies in exploring the generalizability of the TransNetR by testing the proposed algorithm on the out-of-distribution (test distribution is unknown and different from training distribution) dataset. As a use case, we tested our proposed algorithm on the PolypGen (6 unique centers) dataset and two other popular polyp segmentation benchmarking datasets. We obtained state-of-the-art performance on all three datasets during out-of-distribution testing. The source code of TransNetR will be made publicly available at


page 2

page 5

page 13


DilatedSegNet: A Deep Dilated Segmentation Network for Polyp Segmentation

Colorectal cancer (CRC) is the second leading cause of cancer-related de...

TransResU-Net: Transformer based ResU-Net for Real-Time Colonoscopy Polyp Segmentation

Colorectal cancer (CRC) is one of the most common causes of cancer and c...

Automatic Polyp Segmentation with Multiple Kernel Dilated Convolution Network

The detection and removal of precancerous polyps through colonoscopy is ...

Real-Time Target Sound Extraction

We present the first neural network model to achieve real-time and strea...

RUPNet: Residual upsampling network for real-time polyp segmentation

Colorectal cancer is among the most prevalent cause of cancer-related mo...

Automatic Prostate Zonal Segmentation Using Fully Convolutional Network with Feature Pyramid Attention

Our main objective is to develop a novel deep learning-based algorithm f...

Real-Time Polyp Detection, Localisation and Segmentation in Colonoscopy Using Deep Learning

Computer-aided detection, localisation, and segmentation methods can hel...