HiFuse: Hierarchical Multi-Scale Feature Fusion Network for Medical Image Classification

09/21/2022
by   Xiangzuo Huo, et al.
NetEase, Inc
0

Medical image classification has developed rapidly under the impetus of the convolutional neural network (CNN). Due to the fixed size of the receptive field of the convolution kernel, it is difficult to capture the global features of medical images. Although the self-attention-based Transformer can model long-range dependencies, it has high computational complexity and lacks local inductive bias. Much research has demonstrated that global and local features are crucial for image classification. However, medical images have a lot of noisy, scattered features, intra-class variation, and inter-class similarities. This paper proposes a three-branch hierarchical multi-scale feature fusion network structure termed as HiFuse for medical image classification as a new method. It can fuse the advantages of Transformer and CNN from multi-scale hierarchies without destroying the respective modeling so as to improve the classification accuracy of various medical images. A parallel hierarchy of local and global feature blocks is designed to efficiently extract local features and global representations at various semantic scales, with the flexibility to model at different scales and linear computational complexity relevant to image size. Moreover, an adaptive hierarchical feature fusion block (HFF block) is designed to utilize the features obtained at different hierarchical levels comprehensively. The HFF block contains spatial attention, channel attention, residual inverted MLP, and shortcut to adaptively fuse semantic information between various scale features of each branch. The accuracy of our proposed model on the ISIC2018 dataset is 7.6 baseline, 21.5 Compared with other advanced models, the HiFuse model performs the best. Our code is open-source and available from https://github.com/huoxiangzuo/HiFuse.

READ FULL TEXT
06/07/2023

TEC-Net: Vision Transformer Embrace Convolutional Neural Networks for Medical Image Segmentation

The hybrid architecture of convolution neural networks (CNN) and Transfo...
07/04/2019

Multi-Instance Multi-Scale CNN for Medical Image Classification

Deep learning for medical image classification faces three major challen...
03/12/2023

Endoscopy Classification Model Using Swin Transformer and Saliency Map

Endoscopy is a valuable tool for the early diagnosis of colon cancer. Ho...
06/15/2022

How GNNs Facilitate CNNs in Mining Geometric Information from Large-Scale Medical Images

Gigapixel medical images provide massive data, both morphological textur...
12/09/2022

An Attention-based Multi-Scale Feature Learning Network for Multimodal Medical Image Fusion

Medical images play an important role in clinical applications. Multimod...
05/09/2021

Conformer: Local Features Coupling Global Representations for Visual Recognition

Within Convolutional Neural Network (CNN), the convolution operations ar...

Code Repositories

HiFuse

HiFuse: Global and Local Feature Hierarchical Fusion Network for Medical Image Classification


view repo

Please sign up or login with your details

Forgot password? Click here to reset