DeepAI AI Chat
Log In Sign Up

Manipulating Medical Image Translation with Manifold Disentanglement

by   Siyu Liu, et al.

Medical image translation (e.g. CT to MR) is a challenging task as it requires I) faithful translation of domain-invariant features (e.g. shape information of anatomical structures) and II) realistic synthesis of target-domain features (e.g. tissue appearance in MR). In this work, we propose Manifold Disentanglement Generative Adversarial Network (MDGAN), a novel image translation framework that explicitly models these two types of features. It employs a fully convolutional generator to model domain-invariant features, and it uses style codes to separately model target-domain features as a manifold. This design aims to explicitly disentangle domain-invariant features and domain-specific features while gaining individual control of both. The image translation process is formulated as a stylisation task, where the input is "stylised" (translated) into diverse target-domain images based on style codes sampled from the learnt manifold. We test MDGAN for multi-modal medical image translation, where we create two domain-specific manifold clusters on the manifold to translate segmentation maps into pseudo-CT and pseudo-MR images, respectively. We show that by traversing a path across the MR manifold cluster, the target output can be manipulated while still retaining the shape information from the input.


page 1

page 6

page 7

page 9

page 10


Bridging the gap between paired and unpaired medical image translation

Medical image translation has the potential to reduce the imaging worklo...

Distribution Matching Losses Can Hallucinate Features in Medical Image Translation

This paper discusses how distribution matching losses, such as those use...

Empirical Bayesian Mixture Models for Medical Image Translation

Automatically generating one medical imaging modality from another is kn...

MedGAN: Medical Image Translation using GANs

Image-to-image translation is considered a next frontier in the field of...

Deep Manifold Traversal: Changing Labels with Convolutional Features

Many tasks in computer vision can be cast as a "label changing" problem,...

Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation

The image-to-image translation (I2IT) model takes a target label or a re...