Multimodal-Boost: Multimodal Medical Image Super-Resolution using Multi-Attention Network with Wavelet Transform

10/22/2021
by   Farah Deeba, et al.
20

Multimodal medical images are widely used by clinicians and physicians to analyze and retrieve complementary information from high-resolution images in a non-invasive manner. The loss of corresponding image resolution degrades the overall performance of medical image diagnosis. Deep learning based single image super resolution (SISR) algorithms has revolutionized the overall diagnosis framework by continually improving the architectural components and training strategies associated with convolutional neural networks (CNN) on low-resolution images. However, existing work lacks in two ways: i) the SR output produced exhibits poor texture details, and often produce blurred edges, ii) most of the models have been developed for a single modality, hence, require modification to adapt to a new one. This work addresses (i) by proposing generative adversarial network (GAN) with deep multi-attention modules to learn high-frequency information from low-frequency data. Existing approaches based on the GAN have yielded good SR results; however, the texture details of their SR output have been experimentally confirmed to be deficient for medical images particularly. The integration of wavelet transform (WT) and GANs in our proposed SR model addresses the aforementioned limitation concerning textons. The WT divides the LR image into multiple frequency bands, while the transferred GAN utilizes multiple attention and upsample blocks to predict high-frequency components. Moreover, we present a learning technique for training a domain-specific classifier as a perceptual loss function. Combining multi-attention GAN loss with a perceptual loss function results in a reliable and efficient performance. Applying the same model for medical images from diverse modalities is challenging, our work addresses (ii) by training and performing on several modalities via transfer learning.

READ FULL TEXT

page 3

page 4

page 5

page 6

page 7

page 8

page 9

page 13

research
06/15/2019

Single Image Super-resolution via Dense Blended Attention Generative Adversarial Network for Clinical Diagnosis

In clinical diagnosis, doctors are able to see biological tissues and ea...
research
04/20/2021

TWIST-GAN: Towards Wavelet Transform and Transferred GAN for Spatio-Temporal Single Image Super Resolution

Single Image Super-resolution (SISR) produces high-resolution images wit...
research
06/04/2021

SOUP-GAN: Super-Resolution MRI Using Generative Adversarial Networks

There is a growing demand for high-resolution (HR) medical images in bot...
research
11/21/2021

FreqNet: A Frequency-domain Image Super-Resolution Network with Dicrete Cosine Transform

Single image super-resolution(SISR) is an ill-posed problem that aims to...
research
02/21/2021

Tchebichef Transform Domain-based Deep Learning Architecture for Image Super-resolution

The recent outbreak of COVID-19 has motivated researchers to contribute ...
research
07/20/2023

Frequency-aware optical coherence tomography image super-resolution via conditional generative adversarial neural network

Optical coherence tomography (OCT) has stimulated a wide range of medica...
research
03/09/2021

GAN Vocoder: Multi-Resolution Discriminator Is All You Need

Several of the latest GAN-based vocoders show remarkable achievements, o...

Please sign up or login with your details

Forgot password? Click here to reset