Factorised Representation Learning in Cardiac Image Analysis

by   Agisilaos Chartsias, et al.

Typically, a medical image offers spatial information on the anatomy (and pathology) modulated by imaging specific characteristics. Many imaging modalities including Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) can be interpreted in this way. We can venture further and consider that a medical image naturally factors into some spatial factors depicting anatomy and factors that denote the imaging characteristics. Here, we explicitly learn this decomposed (factorised) representation of imaging data, focusing in particular on cardiac images. We propose Spatial Decomposition Network (SDNet), which factorises 2D medical images into spatial anatomical factors and non-spatial imaging factors. We demonstrate that this high-level representation is ideally suited for several medical image analysis tasks, such as semi-supervised segmentation, multi-task segmentation and regression, and image-to-image synthesis. Specifically, we show that our model can match the performance of fully supervised segmentation models, using only a fraction of the labelled images. Critically, we show that our factorised representation also benefits from supervision obtained either when we use auxiliary tasks to train the model in a multi-task setting (e.g. regressing to known cardiac indices), or when aggregating multimodal data from different sources (e.g. pooling together MRI and CT data). To explore the properties of the learned factorisation, we perform latent-space arithmetic and show that we can synthesise CT from MR and vice versa, by swapping the modality factors. We also demonstrate that the factor holding image specific information can be used to predict the input modality with high accuracy.


page 6

page 11

page 13

page 15


Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities

Automatic segmentation of medical images is an important task for many c...

Factorised spatial representation learning: application in semi-supervised myocardial segmentation

The success and generalisation of deep learning algorithms heavily depen...

Disentangle, align and fuse for multimodal and zero-shot image segmentation

Magnetic resonance (MR) protocols rely on several sequences to properly ...

Controllable cardiac synthesis via disentangled anatomy arithmetic

Acquiring annotated data at scale with rare diseases or conditions remai...

4D Semantic Cardiac Magnetic Resonance Image Synthesis on XCAT Anatomical Model

We propose a hybrid controllable image generation method to synthesize a...

Deep generative model-driven multimodal prostate segmentation in radiotherapy

Deep learning has shown unprecedented success in a variety of applicatio...

Dataset Growth in Medical Image Analysis Research

Medical image analysis studies usually require medical image datasets fo...

Please sign up or login with your details

Forgot password? Click here to reset