DeepAI AI Chat
Log In Sign Up

Atlas-ISTN: Joint Segmentation, Registration and Atlas Construction with Image-and-Spatial Transformer Networks

by   Matthew Sinclair, et al.

Deep learning models for semantic segmentation are able to learn powerful representations for pixel-wise predictions, but are sensitive to noise at test time and do not guarantee a plausible topology. Image registration models on the other hand are able to warp known topologies to target images as a means of segmentation, but typically require large amounts of training data, and have not widely been benchmarked against pixel-wise segmentation models. We propose Atlas-ISTN, a framework that jointly learns segmentation and registration on 2D and 3D image data, and constructs a population-derived atlas in the process. Atlas-ISTN learns to segment multiple structures of interest and to register the constructed, topologically consistent atlas labelmap to an intermediate pixel-wise segmentation. Additionally, Atlas-ISTN allows for test time refinement of the model's parameters to optimize the alignment of the atlas labelmap to an intermediate pixel-wise segmentation. This process both mitigates for noise in the target image that can result in spurious pixel-wise predictions, as well as improves upon the one-pass prediction of the model. Benefits of the Atlas-ISTN framework are demonstrated qualitatively and quantitatively on 2D synthetic data and 3D cardiac computed tomography and brain magnetic resonance image data, out-performing both segmentation and registration baseline models. Atlas-ISTN also provides inter-subject correspondence of the structures of interest, enabling population-level shape and motion analysis.


page 5

page 11

page 12

page 14

page 15

page 17

page 21

page 26


DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation

Deep convolutional neural networks (CNNs) are state-of-the-art for seman...

Learning to Segment Medical Images with Scribble-Supervision Alone

Semantic segmentation of medical images is a crucial step for the quanti...

Cardiac Adipose Tissue Segmentation via Image-Level Annotations

Automatically identifying the structural substrates underlying cardiac a...

SegSort: Segmentation by Discriminative Sorting of Segments

Almost all existing deep learning approaches for semantic segmentation t...

Material Segmentation of Multi-View Satellite Imagery

Material recognition methods use image context and local cues for pixel-...

Joint Segmentation and Path Classification of Curvilinear Structures

Detection of curvilinear structures in images has long been of interest....

Fine-grained Image Editing by Pixel-wise Guidance Using Diffusion Models

Generative models, particularly GANs, have been utilized for image editi...