Memory-efficient GAN-based Domain Translation of High Resolution 3D Medical Images

by   Hristina Uzunova, et al.

Generative adversarial networks (GANs) are currently rarely applied on 3D medical images of large size, due to their immense computational demand. The present work proposes a multi-scale patch-based GAN approach for establishing unpaired domain translation by generating 3D medical image volumes of high resolution in a memory-efficient way. The key idea to enable memory-efficient image generation is to first generate a low-resolution version of the image followed by the generation of patches of constant sizes but successively growing resolutions. To avoid patch artifacts and incorporate global information, the patch generation is conditioned on patches from previous resolution scales. Those multi-scale GANs are trained to generate realistically looking images from image sketches in order to perform an unpaired domain translation. This allows to preserve the topology of the test data and generate the appearance of the training domain data. The evaluation of the domain translation scenarios is performed on brain MRIs of size 155x240x240 and thorax CTs of size up to 512x512x512. Compared to common patch-based approaches, the multi-resolution scheme enables better image quality and prevents patch artifacts. Also, it ensures constant GPU memory demand independent from the image size, allowing for the generation of arbitrarily large images.


page 4

page 14

page 16

page 19

page 20

page 21

page 22


Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images

Currently generative adversarial networks (GANs) are rarely applied to m...

Memory Efficient Patch-based Training for INR-based GANs

Recent studies have shown remarkable progress in GANs based on implicit ...

A Streaming Volumetric Image Generation Framework for Development and Evaluation of Out-of-Core Methods

Advances in 3D imaging technology in recent years have allowed for incre...

InfinityGAN: Towards Infinite-Resolution Image Synthesis

We present InfinityGAN, a method to generate arbitrary-resolution images...

Iterative Patch Selection for High-Resolution Image Recognition

High-resolution images are prevalent in various applications, such as au...

Diffusion Models for Memory-efficient Processing of 3D Medical Images

Denoising diffusion models have recently achieved state-of-the-art perfo...

Memory Efficient Invertible Neural Networks for 3D Photoacoustic Imaging

Photoacoustic imaging (PAI) can image high-resolution structures of clin...

Please sign up or login with your details

Forgot password? Click here to reset