Learning a Spatio-Temporal Embedding for Video Instance Segmentation

12/19/2019
by   Anthony Hu, et al.
13

We present a novel embedding approach for video instance segmentation. Our method learns a spatio-temporal embedding integrating cues from appearance, motion, and geometry; a 3D causal convolutional network models motion, and a monocular self-supervised depth loss models geometry. In this embedding space, video-pixels of the same instance are clustered together while being separated from other instances, to naturally track instances over time without any complex post-processing. Our network runs in real-time as our architecture is entirely causal - we do not incorporate information from future frames, contrary to previous methods. We show that our model can accurately track and segment instances, even with occlusions and missed detections, advancing the state-of-the-art on the KITTI Multi-Object and Tracking Dataset.

READ FULL TEXT

page 1

page 9

page 13

page 14

page 15

page 16

page 17

research
03/18/2020

STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos

Existing methods for instance segmentation in videos typically involve m...
research
06/06/2018

Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks

Different to semantic segmentation, instance segmentation assigns unique...
research
12/15/2022

Solve the Puzzle of Instance Segmentation in Videos: A Weakly Supervised Framework with Spatio-Temporal Collaboration

Instance segmentation in videos, which aims to segment and track multipl...
research
04/19/2022

Less than Few: Self-Shot Video Instance Segmentation

The goal of this paper is to bypass the need for labelled examples in fe...
research
08/28/2023

VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation

Existing approaches to unsupervised video instance segmentation typicall...
research
04/11/2019

MAIN: Multi-Attention Instance Network for Video Segmentation

Instance-level video segmentation requires a solid integration of spatia...
research
04/01/2021

Learning to Track Instances without Video Annotations

Tracking segmentation masks of multiple instances has been intensively s...

Please sign up or login with your details

Forgot password? Click here to reset