Fast Video Object Segmentation via Mask Transfer Network

08/28/2019
by   Tao Zhuo, et al.
5

Accuracy and processing speed are two important factors that affect the use of video object segmentation (VOS) in real applications. With the advanced techniques of deep neural networks, the accuracy has been significantly improved, however, the speed is still far below the real-time needs because of the complicated network design, such as the requirement of the first frame fine-tuning step. To overcome this limitation, we propose a novel mask transfer network (MTN), which can greatly boost the processing speed of VOS and also achieve a reasonable accuracy. The basic idea of MTN is to transfer the reference mask to the target frame via an efficient global pixel matching strategy. The global pixel matching between the reference frame and the target frame is to ensure good matching results. To enhance the matching speed, we perform the matching on a downsampled feature map with 1/32 of the original frame size. At the same time, to preserve the detailed mask information in such a small feature map, a mask network is designed to encode the annotated mask information with 512 channels. Finally, an efficient feature warping method is used to transfer the encoded reference mask to the target frame. Based on this design, our method avoids the fine-tuning step on the first frame and does not rely on the temporal cues and particular object categories. Therefore, it runs very fast and can be conveniently trained only with images, as well as being robust to unseen objects. Experiments on the DAVIS datasets demonstrate that MTN can achieve a speed of 37 fps, and also shows a competitive accuracy in comparison to the state-of-the-art methods.

READ FULL TEXT

page 3

page 8

research
02/25/2019

FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation

Many of the recent successful methods for video object segmentation (VOS...
research
07/09/2021

Fast Pixel-Matching for Video Object Segmentation

Video object segmentation, aiming to segment the foreground objects give...
research
09/04/2022

Pixel-Level Equalized Matching for Video Object Segmentation

Feature similarity matching, which transfers the information of the refe...
research
02/17/2020

Directional Deep Embedding and Appearance Learning for Fast Video Object Segmentation

Most recent semi-supervised video object segmentation (VOS) methods rely...
research
09/27/2019

DMM-Net: Differentiable Mask-Matching Network for Video Object Segmentation

In this paper, we propose the differentiable mask-matching network (DMM-...
research
12/23/2021

Iteratively Selecting an Easy Reference Frame Makes Unsupervised Video Object Segmentation Easier

Unsupervised video object segmentation (UVOS) is a per-pixel binary labe...
research
03/21/2018

Video Object Segmentation with Language Referring Expressions

Most state-of-the-art semi-supervised video object segmentation methods ...

Please sign up or login with your details

Forgot password? Click here to reset