Region Aware Video Object Segmentation with Deep Motion Modeling

07/21/2022
by   Bo Miao, et al.
1

Current semi-supervised video object segmentation (VOS) methods usually leverage the entire features of one frame to predict object masks and update memory. This introduces significant redundant computations. To reduce redundancy, we present a Region Aware Video Object Segmentation (RAVOS) approach that predicts regions of interest (ROIs) for efficient object segmentation and memory storage. RAVOS includes a fast object motion tracker to predict their ROIs in the next frame. For efficient segmentation, object features are extracted according to the ROIs, and an object decoder is designed for object-level segmentation. For efficient memory storage, we propose motion path memory to filter out redundant context by memorizing the features within the motion path of objects between two frames. Besides RAVOS, we also propose a large-scale dataset, dubbed OVOS, to benchmark the performance of VOS models under occlusions. Evaluation on DAVIS and YouTube-VOS benchmarks and our new OVOS dataset show that our method achieves state-of-the-art performance with significantly faster inference time, e.g., 86.1 J F at 42 FPS on DAVIS and 84.4 J F at 23 FPS on YouTube-VOS.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 8

research
07/27/2021

Self-Supervised Video Object Segmentation by Motion-Aware Mask Propagation

We propose a self-supervised spatio-temporal matching method coined Moti...
research
08/03/2022

Per-Clip Video Object Segmentation

Recently, memory-based approaches show promising results on semi-supervi...
research
07/26/2021

Efficient Video Object Segmentation with Compressed Video

We propose an efficient inference framework for semi-supervised video ob...
research
12/16/2021

HODOR: High-level Object Descriptors for Object Re-segmentation in Video Learned from Static Images

Existing state-of-the-art methods for Video Object Segmentation (VOS) le...
research
02/09/2021

SwiftNet: Real-time Video Object Segmentation

In this work we present SwiftNet for real-time semi-supervised video obj...
research
08/22/2022

SWEM: Towards Real-Time Video Object Segmentation with Sequential Weighted Expectation-Maximization

Matching-based methods, especially those based on space-time memory, are...
research
03/14/2023

InstMove: Instance Motion for Object-centric Video Segmentation

Despite significant efforts, cutting-edge video segmentation methods sti...

Please sign up or login with your details

Forgot password? Click here to reset