DeepAI AI Chat
Log In Sign Up

Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks

by   Christian Payer, et al.

Different to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same class. In this work, we propose a novel recurrent fully convolutional network architecture for tracking such instance segmentations over time. The network architecture incorporates convolutional gated recurrent units (ConvGRU) into a stacked hourglass network to utilize temporal video information. Furthermore, we train the network with a novel embedding loss based on cosine similarities, such that the network predicts unique embeddings for every instance throughout videos. Afterwards, these embeddings are clustered among subsequent video frames to create the final tracked instance segmentations. We evaluate the recurrent hourglass network by segmenting left ventricles in MR videos of the heart, where it outperforms a network that does not incorporate video information. Furthermore, we show applicability of the cosine embedding loss for segmenting leaf instances on still images of plants. Finally, we evaluate the framework for instance segmentation and tracking on six datasets of the ISBI celltracking challenge, where it shows state-of-the-art performance.


page 2

page 6

page 11

page 12


Convolutional Gated Recurrent Networks for Video Segmentation

Semantic segmentation has recently witnessed major progress, where fully...

Learning a Spatio-Temporal Embedding for Video Instance Segmentation

We present a novel embedding approach for video instance segmentation. O...

3D Graph Embedding Learning with a Structure-aware Loss Function for Point Cloud Semantic Instance Segmentation

This paper introduces a novel approach for 3D semantic instance segmenta...

Learning Video Instance Segmentation with Recurrent Graph Neural Networks

Most existing approaches to video instance segmentation comprise multipl...

VoxelEmbed: 3D Instance Segmentation and Tracking with Voxel Embedding based Deep Learning

Recent advances in bioimaging have provided scientists a superior high s...

Recurrent Pixel Embedding for Instance Grouping

We introduce a differentiable, end-to-end trainable framework for solvin...

US-net for robust and efficient nuclei instance segmentation

We present a novel neural network architecture, US-Net, for robust nucle...