Make One-Shot Video Object Segmentation Efficient Again

12/03/2020
by   Tim Meinhardt, et al.
0

Video object segmentation (VOS) describes the task of segmenting a set of objects in each frame of a video. In the semi-supervised setting, the first mask of each object is provided at test time. Following the one-shot principle, fine-tuning VOS methods train a segmentation model separately on each given object mask. However, recently the VOS community has deemed such a test time optimization and its impact on the test runtime as unfeasible. To mitigate the inefficiencies of previous fine-tuning approaches, we present efficient One-Shot Video Object Segmentation (e-OSVOS). In contrast to most VOS approaches, e-OSVOS decouples the object detection task and predicts only local segmentation masks by applying a modified version of Mask R-CNN. The one-shot test runtime and performance are optimized without a laborious and handcrafted hyperparameter search. To this end, we meta learn the model initialization and learning rates for the test time optimization. To achieve optimal learning behavior, we predict individual learning rates at a neuron level. Furthermore, we apply an online adaptation to address the common performance degradation throughout a sequence by continuously fine-tuning the model on previous mask predictions supported by a frame-to-frame bounding box propagation. e-OSVOS provides state-of-the-art results on DAVIS 2016, DAVIS 2017, and YouTube-VOS for one-shot fine-tuning methods while reducing the test runtime substantially. Code is available at https://github.com/dvl-tum/e-osvos.

READ FULL TEXT

page 5

page 11

04/09/2019

BoLTVOS: Box-Level Tracking for Video Object Segmentation

We approach video object segmentation (VOS) by splitting the task into t...
03/28/2019

Fast video object segmentation with Spatio-Temporal GANs

Learning descriptive spatio-temporal object models from data is paramoun...
09/04/2018

VideoMatch: Matching based Video Object Segmentation

Video object segmentation is challenging yet important in a wide variety...
11/13/2020

Image Animation with Perturbed Masks

We present a novel approach for image-animation of a source image by a d...
06/28/2017

Online Adaptation of Convolutional Neural Networks for Video Object Segmentation

We tackle the task of semi-supervised video object segmentation, i.e. se...
09/27/2019

DMM-Net: Differentiable Mask-Matching Network for Video Object Segmentation

In this paper, we propose the differentiable mask-matching network (DMM-...
03/18/2021

GPT Understands, Too

While GPTs with traditional fine-tuning fail to achieve strong results o...