Spacetime Graph Optimization for Video Object Segmentation

07/07/2019
by   Emanuela Haller, et al.
4

In this paper we address the challenging task of object discovery and segmentation in video. We introduce an efficient method that can be applied in supervised and unsupervised scenarios, using a graph-based representation in both space and time. Our method exploits the consistency in appearance and motion patterns of pixels belonging to the same object. We formulate the task as a clustering problem: graph nodes at the pixel level that belong to the object of interest should form a strong cluster, linked through long range optical flow chains and with similar motion and appearance features along those chains. On one hand, the optimization problem aims to maximize the segmentation clustering score based on the structure of pixel motions through space and time. On the other, the segmentation should be consistent with the features at the level of nodes, s.t. these features should be able to predict the segmentation labels. The solution to our problem relates to spectral clustering as well as to the classical regression analysis. It leads to a fast algorithm that converges in a few iterations to a global optimum of the relaxed problem, using fixed point iteration. The proposed method, namely GO-VOS, is relatively fast and accurate. It can be used both as a standalone and completely unsupervised method or in combination with other segmentation methods. In experiments, we demonstrate top performance on several challenging datasets: DAVIS, SegTrack and YouTube-Objects.

READ FULL TEXT

page 5

page 6

page 8

research
07/05/2019

A Spectral Approach to Unsupervised Object Segmentation in Video

We formulate object segmentation in video as a graph partitioning proble...
research
12/13/2020

Iterative Knowledge Exchange Between Deep Learning and Space-Time Spectral Clustering for Unsupervised Segmentation in Videos

We propose a dual system for unsupervised object segmentation in video, ...
research
12/15/2022

Learning a Fast 3D Spectral Approach to Object Segmentation and Tracking over Space and Time

We pose video object segmentation as spectral graph clustering in space ...
research
12/06/2018

Object Discovery in Videos as Foreground Motion Clustering

We consider the problem of providing dense segmentation masks for object...
research
03/14/2023

InstMove: Instance Motion for Object-centric Video Segmentation

Despite significant efforts, cutting-edge video segmentation methods sti...
research
09/04/2022

Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentation

Unsupervised video object segmentation (VOS) aims to detect the most sal...
research
09/04/2018

Unsupervised Video Object Segmentation using Motion Saliency-Guided Spatio-Temporal Propagation

Unsupervised video segmentation plays an important role in a wide variet...

Please sign up or login with your details

Forgot password? Click here to reset