Contextual Guided Segmentation Framework for Semi-supervised Video Instance Segmentation

06/07/2021
by   Trung-Nghia Le, et al.
0

In this paper, we propose Contextual Guided Segmentation (CGS) framework for video instance segmentation in three passes. In the first pass, i.e., preview segmentation, we propose Instance Re-Identification Flow to estimate main properties of each instance (i.e., human/non-human, rigid/deformable, known/unknown category) by propagating its preview mask to other frames. In the second pass, i.e., contextual segmentation, we introduce multiple contextual segmentation schemes. For human instance, we develop skeleton-guided segmentation in a frame along with object flow to correct and refine the result across frames. For non-human instance, if the instance has a wide variation in appearance and belongs to known categories (which can be inferred from the initial mask), we adopt instance segmentation. If the non-human instance is nearly rigid, we train FCNs on synthesized images from the first frame of a video sequence. In the final pass, i.e., guided segmentation, we develop a novel fined-grained segmentation method on non-rectangular regions of interest (ROIs). The natural-shaped ROI is generated by applying guided attention from the neighbor frames of the current one to reduce the ambiguity in the segmentation of different overlapping instances. Forward mask propagation is followed by backward mask propagation to further restore missing instance fragments due to re-appeared instances, fast motion, occlusion, or heavy deformation. Finally, instances in each frame are merged based on their depth values, together with human and non-human object interaction and rare instance priority. Experiments conducted on the DAVIS Test-Challenge dataset demonstrate the effectiveness of our proposed framework. We achieved the 3rd consistently in the DAVIS Challenges 2017-2019 with 75.4 global score, region similarity, and contour accuracy, respectively.

READ FULL TEXT

page 2

page 7

page 8

page 9

page 10

page 11

page 14

research
10/24/2018

Mask Propagation Network for Video Object Segmentation

In this work, we propose a mask propagation network to treat the video s...
research
12/10/2019

Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagation

We introduce a method for simultaneously classifying, segmenting and tra...
research
05/24/2019

OVSNet : Towards One-Pass Real-Time Video Object Segmentation

Video object segmentation aims at accurately segmenting the target objec...
research
04/17/2019

MHP-VOS: Multiple Hypotheses Propagation for Video Object Segmentation

We address the problem of semi-supervised video object segmentation (VOS...
research
03/31/2022

Human Instance Segmentation and Tracking via Data Association and Single-stage Detector

Human video instance segmentation plays an important role in computer un...
research
04/22/2022

Tag-Based Attention Guided Bottom-Up Approach for Video Instance Segmentation

Video Instance Segmentation is a fundamental computer vision task that d...
research
04/15/2020

A Transductive Approach for Video Object Segmentation

Semi-supervised video object segmentation aims to separate a target obje...

Please sign up or login with your details

Forgot password? Click here to reset