Motion-Appearance Interactive Encoding for Object Segmentation in Unconstrained Videos

07/25/2017
by   Chunchao Guo, et al.
0

We present a novel method of integrating motion and appearance cues for foreground object segmentation in unconstrained videos. Unlike conventional methods encoding motion and appearance patterns individually, our method puts particular emphasis on their mutual assistance. Specifically, we propose using an interactively constrained encoding (ICE) scheme to incorporate motion and appearance patterns into a graph that leads to a spatiotemporal energy optimization. The reason of utilizing ICE is that both motion and appearance cues for the same target share underlying correlative structure, thus can be exploited in a deeply collaborative manner. We perform ICE not only in the initialization but also in the refinement stage of a two-layer framework for object segmentation. This scheme allows our method to consistently capture structural patterns about object perceptions throughout the whole framework. Our method can be operated on superpixels instead of raw pixels to reduce the number of graph nodes by two orders of magnitude. Moreover, we propose to partially explore the multi-object localization problem with inter-occlusion by weighted bipartite graph matching. Comprehensive experiments on three benchmark datasets (i.e., SegTrack, MOViCS, and GaTech) demonstrate the effectiveness of our approach compared with extensive state-of-the-art methods.

READ FULL TEXT

page 2

page 4

page 5

page 8

page 9

page 10

research
12/17/2022

Improving Unsupervised Video Object Segmentation with Motion-Appearance Synergy

We present IMAS, a method that segments the primary objects in videos wi...
research
09/04/2022

Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentation

Unsupervised video object segmentation (VOS) aims to detect the most sal...
research
12/06/2018

Object Discovery in Videos as Foreground Motion Clustering

We consider the problem of providing dense segmentation masks for object...
research
09/13/2018

Seeing Tree Structure from Vibration

Humans recognize object structure from both their appearance and motion;...
research
11/22/2022

Domain Alignment and Temporal Aggregation for Unsupervised Video Object Segmentation

Unsupervised video object segmentation aims at detecting and segmenting ...
research
11/04/2014

Simultaneous Localization, Mapping, and Manipulation for Unsupervised Object Discovery

We present an unsupervised framework for simultaneous appearance-based o...
research
03/21/2020

Appearance Fusion of Multiple Cues for Video Co-localization

This work addresses a problem named video co-localization that aims at l...

Please sign up or login with your details

Forgot password? Click here to reset