DeepAI AI Chat
Log In Sign Up

Manipulating Medical Image Translation with Manifold Disentanglement

11/27/2020
by   Siyu Liu, et al.
23

Medical image translation (e.g. CT to MR) is a challenging task as it requires I) faithful translation of domain-invariant features (e.g. shape information of anatomical structures) and II) realistic synthesis of target-domain features (e.g. tissue appearance in MR). In this work, we propose Manifold Disentanglement Generative Adversarial Network (MDGAN), a novel image translation framework that explicitly models these two types of features. It employs a fully convolutional generator to model domain-invariant features, and it uses style codes to separately model target-domain features as a manifold. This design aims to explicitly disentangle domain-invariant features and domain-specific features while gaining individual control of both. The image translation process is formulated as a stylisation task, where the input is "stylised" (translated) into diverse target-domain images based on style codes sampled from the learnt manifold. We test MDGAN for multi-modal medical image translation, where we create two domain-specific manifold clusters on the manifold to translate segmentation maps into pseudo-CT and pseudo-MR images, respectively. We show that by traversing a path across the MR manifold cluster, the target output can be manipulated while still retaining the shape information from the input.

READ FULL TEXT

page 1

page 6

page 7

page 9

page 10

10/15/2021

Bridging the gap between paired and unpaired medical image translation

Medical image translation has the potential to reduce the imaging worklo...
05/22/2018

Distribution Matching Losses Can Hallucinate Features in Medical Image Translation

This paper discusses how distribution matching losses, such as those use...
08/16/2019

Empirical Bayesian Mixture Models for Medical Image Translation

Automatically generating one medical imaging modality from another is kn...
06/17/2018

MedGAN: Medical Image Translation using GANs

Image-to-image translation is considered a next frontier in the field of...
11/19/2015

Deep Manifold Traversal: Changing Labels with Convolutional Features

Many tasks in computer vision can be cast as a "label changing" problem,...
10/11/2021

Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation

The image-to-image translation (I2IT) model takes a target label or a re...