Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motion

05/16/2022
by   Subhabrata Choudhury, et al.
15

Motion, measured via optical flow, provides a powerful cue to discover and learn objects in images and videos. However, compared to using appearance, it has some blind spots, such as the fact that objects become invisible if they do not move. In this work, we propose an approach that combines the strengths of motion-based and appearance-based segmentation. We propose to supervise an image segmentation network, tasking it with predicting regions that are likely to contain simple motion patterns, and thus likely to correspond to objects. We apply this network in two modes. In the unsupervised video segmentation mode, the network is trained on a collection of unlabelled videos, using the learning process itself as an algorithm to segment these videos. In the unsupervised image segmentation model, the network is learned using videos and applied to segment independent still images. With this, we obtain strong empirical results in unsupervised video and image segmentation, significantly outperforming the state of the art on benchmarks such as DAVIS, sometimes with a 5% IoU gap.

READ FULL TEXT

page 7

page 8

page 10

page 21

page 22

page 23

page 24

page 25

research
10/21/2022

Unsupervised Multi-object Segmentation by Predicting Probable Motion Patterns

We propose a new approach to learn to segment multiple image objects wit...
research
06/10/2021

Unsupervised Co-part Segmentation through Assembly

Co-part segmentation is an important problem in computer vision for its ...
research
09/22/2011

Detachable Object Detection: Segmentation and Depth Ordering From Short-Baseline Video

We describe an approach for segmenting an image into regions that corres...
research
04/17/2023

Bootstrapping Objectness from Videos by Relaxed Common Fate and Visual Grouping

We study learning object segmentation from unlabeled videos. Humans can ...
research
11/11/2021

The Emergence of Objectness: Learning Zero-Shot Segmentation from Videos

Humans can easily segment moving objects without knowing what they are. ...
research
10/17/2021

Temporally stable video segmentation without video annotations

Temporally consistent dense video annotations are scarce and hard to col...
research
09/03/2016

Towards Segmenting Consumer Stereo Videos: Benchmark, Baselines and Ensembles

Are we ready to segment consumer stereo videos? The amount of this data ...

Please sign up or login with your details

Forgot password? Click here to reset