Self-supervised Training of Proposal-based Segmentation via Background Prediction

by   Isinsu Katircioglu, et al.

While supervised object detection methods achieve impressive accuracy, they generalize poorly to images whose appearance significantly differs from the data they have been trained on. To address this in scenarios where annotating data is prohibitively expensive, we introduce a self-supervised approach to object detection and segmentation, able to work with monocular images captured with a moving camera. At the heart of our approach lies the observation that segmentation and background reconstruction are linked tasks, and the idea that, because we observe a structured scene, background regions can be re-synthesized from their surroundings, whereas regions depicting the object cannot. We therefore encode this intuition as a self-supervised loss function that we exploit to train a proposal-based segmentation network. To account for the discrete nature of object proposals, we develop a Monte Carlo-based training strategy that allows us to explore the large space of object proposals. Our experiments demonstrate that our approach yields accurate detections and segmentations in images that visually depart from those of standard benchmarks, outperforming existing self-supervised methods and approaching weakly supervised ones that exploit large annotated datasets.


page 2

page 6

page 7


Self-supervised Segmentation via Background Inpainting

While supervised object detection and segmentation methods achieve impre...

Object-Aware Cropping for Self-Supervised Learning

A core component of the recent success of self-supervised learning is cr...

Self-Supervised Video Object Segmentation via Cutout Prediction and Tagging

We propose a novel self-supervised Video Object Segmentation (VOS) appro...

Self-supervised Human Detection and Segmentation via Multi-view Consensus

Self-supervised detection and segmentation of foreground objects in comp...

SCOPS: Self-Supervised Co-Part Segmentation

Parts provide a good intermediate representation of objects that is robu...

Towards Human-Machine Cooperation: Self-supervised Sample Mining for Object Detection

Though quite challenging, leveraging large-scale unlabeled or partially ...

Please sign up or login with your details

Forgot password? Click here to reset