Learning to Better Segment Objects from Unseen Classes with Unlabeled Videos

04/25/2021
by   Yuming Du, et al.
7

The ability to localize and segment objects from unseen classes would open the door to new applications, such as autonomous object learning in active vision. Nonetheless, improving the performance on unseen classes requires additional training data, while manually annotating the objects of the unseen classes can be labor-extensive and expensive. In this paper, we explore the use of unlabeled video sequences to automatically generate training data for objects of unseen classes. It is in principle possible to apply existing video segmentation methods to unlabeled videos and automatically obtain object masks, which can then be used as a training set even for classes with no manual labels available. However, our experiments show that these methods do not perform well enough for this purpose. We therefore introduce a Bayesian method that is specifically designed to automatically create such a training set: Our method starts from a set of object proposals and relies on (non-realistic) analysis-by-synthesis to select the correct ones by performing an efficient optimization over all the frames simultaneously. Through extensive experiments, we show that our method can generate a high-quality training set which significantly boosts the performance of segmenting objects of unseen classes. We thus believe that our method could open the door for open-world instance segmentation using abundant Internet videos.

READ FULL TEXT

page 2

page 5

page 8

page 15

page 17

research
04/04/2023

Towards Open-Vocabulary Video Instance Segmentation

Video Instance Segmentation(VIS) aims at segmenting and categorizing obj...
research
05/26/2023

Towards Open-World Segmentation of Parts

Segmenting object parts such as cup handles and animal bodies is importa...
research
01/26/2019

4D Generic Video Object Proposals

Many high-level video understanding methods require input in the form of...
research
07/06/2022

Two-stage Decision Improves Open-Set Panoptic Segmentation

Open-set panoptic segmentation (OPS) problem is a new research direction...
research
04/12/2023

Impact of Pseudo Depth on Open World Object Segmentation with Minimal User Guidance

Pseudo depth maps are depth map predicitions which are used as ground tr...
research
08/17/2022

Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuning

Scene graph generation (SGG) is a fundamental task aimed at detecting vi...
research
09/12/2022

Holistic Segmentation

As panoptic segmentation provides a prediction for every pixel in input,...

Please sign up or login with your details

Forgot password? Click here to reset