Leveraging Motion Priors in Videos for Improving Human Segmentation

07/30/2018
by   Yu-Ting Chen, et al.
2

Despite many advances in deep-learning based semantic segmentation, performance drop due to distribution mismatch is often encountered in the real world. Recently, a few domain adaptation and active learning approaches have been proposed to mitigate the performance drop. However, very little attention has been made toward leveraging information in videos which are naturally captured in most camera systems. In this work, we propose to leverage "motion prior" in videos for improving human segmentation in a weakly-supervised active learning setting. By extracting motion information using optical flow in videos, we can extract candidate foreground motion segments (referred to as motion prior) potentially corresponding to human segments. We propose to learn a memory-network-based policy model to select strong candidate segments (referred to as strong motion prior) through reinforcement learning. The selected segments have high precision and are directly used to finetune the model. In a newly collected surveillance camera dataset and a publicly available UrbanStreet dataset, our proposed method improves the performance of human segmentation across multiple scenes and modalities (i.e., RGB to Infrared (IR)). Last but not least, our method is empirically complementary to existing domain adaptation approaches such that additional performance gain is achieved by combining our weakly-supervised active learning approach with domain adaptation approaches.

READ FULL TEXT

page 2

page 8

page 11

page 14

research
04/22/2021

Domain Adaptation for Semantic Segmentation via Patch-Wise Contrastive Learning

We introduce a novel approach to unsupervised and semi-supervised domain...
research
09/21/2023

MoDA: Leveraging Motion Priors from Videos for Advancing Unsupervised Domain Adaptation in Semantic Segmentation

Unsupervised domain adaptation (UDA) is an effective approach to handle ...
research
09/15/2022

Active Self-Training for Weakly Supervised 3D Scene Semantic Segmentation

Since the preparation of labeled data for training semantic segmentation...
research
01/30/2021

DRIV100: In-The-Wild Multi-Domain Dataset and Evaluation for Real-World Domain Adaptation of Semantic Segmentation

Together with the recent advances in semantic segmentation, many domain ...
research
01/02/2017

Weakly Supervised Semantic Segmentation using Web-Crawled Videos

We propose a novel algorithm for weakly supervised semantic segmentation...
research
06/19/2015

Crowd Flow Segmentation in Compressed Domain using CRF

Crowd flow segmentation is an important step in many video surveillance ...
research
04/14/2023

Spectral Transfer Guided Active Domain Adaptation For Thermal Imagery

The exploitation of visible spectrum datasets has led deep networks to s...

Please sign up or login with your details

Forgot password? Click here to reset