UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation

07/02/2021
by   Yunhe Gao, et al.
21

Transformer architecture has emerged to be successful in a number of natural language processing tasks. However, its applications to medical vision remain largely unexplored. In this study, we present UTNet, a simple yet powerful hybrid Transformer architecture that integrates self-attention into a convolutional neural network for enhancing medical image segmentation. UTNet applies self-attention modules in both encoder and decoder for capturing long-range dependency at different scales with minimal overhead. To this end, we propose an efficient self-attention mechanism along with relative position encoding that reduces the complexity of self-attention operation significantly from O(n^2) to approximate O(n). A new self-attention decoder is also proposed to recover fine-grained details from the skipped connections in the encoder. Our approach addresses the dilemma that Transformer requires huge amounts of data to learn vision inductive bias. Our hybrid layer design allows the initialization of Transformer into convolutional networks without a need of pre-training. We have evaluated UTNet on the multi-label, multi-vendor cardiac magnetic resonance imaging cohort. UTNet demonstrates superior segmentation performance and robustness against the state-of-the-art approaches, holding the promise to generalize well on other medical image segmentations.

READ FULL TEXT
research
05/15/2023

MaxViT-UNet: Multi-Axis Attention for Medical Image Segmentation

Convolutional neural networks have made significant strides in medical i...
research
02/28/2022

A Multi-scale Transformer for Medical Image Segmentation: Architectures, Model Efficiency, and Benchmarks

Transformers have emerged to be successful in a number of natural langua...
research
11/17/2022

Parameter-Efficient Transformer with Hybrid Axial-Attention for Medical Image Segmentation

Transformers have achieved remarkable success in medical image analysis ...
research
10/27/2022

UNet-2022: Exploring Dynamics in Non-isomorphic Architecture

Recent medical image segmentation models are mostly hybrid, which integr...
research
10/12/2021

MEDUSA: Multi-scale Encoder-Decoder Self-Attention Deep Neural Network Architecture for Medical Image Analysis

Medical image analysis continues to hold interesting challenges given th...
research
02/19/2023

MedViT: A Robust Vision Transformer for Generalized Medical Image Classification

Convolutional Neural Networks (CNNs) have advanced existing medical syst...
research
06/29/2022

The Lighter The Better: Rethinking Transformers in Medical Image Segmentation Through Adaptive Pruning

Vision transformers have recently set off a new wave in the field of med...

Please sign up or login with your details

Forgot password? Click here to reset