Convolutional Gated Recurrent Networks for Video Segmentation

11/16/2016
by   Mennatullah Siam, et al.
0

Semantic segmentation has recently witnessed major progress, where fully convolutional neural networks have shown to perform well. However, most of the previous work focused on improving single image segmentation. To our knowledge, no prior work has made use of temporal video information in a recurrent network. In this paper, we introduce a novel approach to implicitly utilize temporal data in videos for online semantic segmentation. The method relies on a fully convolutional network that is embedded into a gated recurrent architecture. This design receives a sequence of consecutive video frames and outputs the segmentation of the last frame. Convolutional gated recurrent networks are used for the recurrent part to preserve spatial connectivities in the image. Our proposed method can be applied in both online and batch segmentation. This architecture is tested for both binary and semantic video segmentation tasks. Experiments are conducted on the recent benchmarks in SegTrack V2, Davis, CityScapes, and Synthia. Using recurrent fully convolutional networks improved the baseline network performance in all of our experiments. Namely, 5 respectively, 5.7 categorical mean IoU in CityScapes. The performance of the RFCN network depends on its baseline fully convolutional network. Thus RFCN architecture can be seen as a method to improve its baseline segmentation network by exploiting spatiotemporal information in videos.

READ FULL TEXT

page 1

page 4

page 6

page 7

research
06/01/2016

Recurrent Fully Convolutional Networks for Video Segmentation

Image segmentation is an important step in most visual tasks. While conv...
research
08/29/2019

Exploiting Temporality for Semi-Supervised Video Segmentation

In recent years, there has been remarkable progress in supervised image ...
research
06/06/2018

Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks

Different to semantic segmentation, instance segmentation assigns unique...
research
06/24/2018

Dilated Temporal Fully-Convolutional Network for Semantic Segmentation of Motion Capture Data

Semantic segmentation of motion capture sequences plays a key part in ma...
research
08/01/2018

Recurrent neural networks for aortic image sequence segmentation with sparse annotations

Segmentation of image sequences is an important task in medical image an...
research
02/02/2023

FCB-SwinV2 Transformer for Polyp Segmentation

Polyp segmentation within colonoscopy video frames using deep learning m...
research
11/27/2018

Beyond One Glance: Gated Recurrent Architecture for Hand Segmentation

As mixed reality is gaining increased momentum, the development of effec...

Please sign up or login with your details

Forgot password? Click here to reset