MHP-VOS: Multiple Hypotheses Propagation for Video Object Segmentation

by   Shuangjie Xu, et al.

We address the problem of semi-supervised video object segmentation (VOS), where the masks of objects of interests are given in the first frame of an input video. To deal with challenging cases where objects are occluded or missing, previous work relies on greedy data association strategies that make decisions for each frame individually. In this paper, we propose a novel approach to defer the decision making for a target object in each frame, until a global view can be established with the entire video being taken into consideration. Our approach is in the same spirit as Multiple Hypotheses Tracking (MHT) methods, making several critical adaptations for the VOS problem. We employ the bounding box (bbox) hypothesis for tracking tree formation, and the multiple hypotheses are spawned by propagating the preceding bbox into the detected bbox proposals within a gated region starting from the initial object mask in the first frame. The gated region is determined by a gating scheme which takes into account a more comprehensive motion model rather than the simple Kalman filtering model in traditional MHT. To further design more customized algorithms tailored for VOS, we develop a novel mask propagation score instead of the appearance similarity score that could be brittle due to large deformations. The mask propagation score, together with the motion score, determines the affinity between the hypotheses during tree pruning. Finally, a novel mask merging strategy is employed to handle mask conflicts between objects. Extensive experiments on challenging datasets demonstrate the effectiveness of the proposed method, especially in the case of object missing.


page 1

page 4

page 6

page 7

page 8


Mask Propagation Network for Video Object Segmentation

In this work, we propose a mask propagation network to treat the video s...

PReMVOS: Proposal-generation, Refinement and Merging for Video Object Segmentation

We address semi-supervised video object segmentation, the task of automa...

Contextual Guided Segmentation Framework for Semi-supervised Video Instance Segmentation

In this paper, we propose Contextual Guided Segmentation (CGS) framework...

Discriminative Online Learning for Fast Video Object Segmentation

We address the highly challenging problem of video object segmentation. ...

Saliency based Semi-supervised Learning for Orbiting Satellite Tracking

The trajectory and boundary of an orbiting satellite are fundamental inf...

Video Object Segmentation with Joint Re-identification and Attention-Aware Mask Propagation

The problem of video object segmentation can become extremely challengin...

Click Carving: Segmenting Objects in Video with Point Clicks

We present a novel form of interactive video object segmentation where a...

Please sign up or login with your details

Forgot password? Click here to reset