Multi-stream CNN based Video Semantic Segmentation for Automated Driving

01/08/2019
by   Ganesh Sistu, et al.
12

Majority of semantic segmentation algorithms operate on a single frame even in the case of videos. In this work, the goal is to exploit temporal information within the algorithm model for leveraging motion cues and temporal consistency. We propose two simple high-level architectures based on Recurrent FCN (RFCN) and Multi-Stream FCN (MSFCN) networks. In case of RFCN, a recurrent network namely LSTM is inserted between the encoder and decoder. MSFCN combines the encoders of different frames into a fused encoder via 1x1 channel-wise convolution. We use a ResNet50 network as the baseline encoder and construct three networks namely MSFCN of order 2 & 3 and RFCN of order 2. MSFCN-3 produces the best results with an accuracy improvement of 9 Highway and New York-like city scenarios in the SYNTHIA-CVPR'16 dataset using mean IoU metric. MSFCN-3 also produced 11 datasets over the baseline FCN network. We also designed an efficient version of MSFCN-2 and RFCN-2 using weight sharing among the two encoders. The efficient MSFCN-2 provided an improvement of 11 with negligible increase in computational complexity compared to the baseline version.

READ FULL TEXT

page 3

page 6

page 7

research
05/03/2019

Semantic Segmentation of Video Sequences with Convolutional LSTMs

Most of the semantic segmentation approaches have been developed for sin...
research
11/29/2020

UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information

Semantic segmentation of aerial videos has been extensively used for dec...
research
08/21/2016

STFCN: Spatio-Temporal FCN for Semantic Video Segmentation

This paper presents a novel method to involve both spatial and temporal ...
research
01/19/2019

Design of Real-time Semantic Segmentation Decoder for Automated Driving

Semantic segmentation remains a computationally intensive algorithm for ...
research
10/02/2019

Using Image Priors to Improve Scene Understanding

Semantic segmentation algorithms that can robustly segment objects acros...
research
07/18/2018

Video Time: Properties, Encoders and Evaluation

Time-aware encoding of frame sequences in a video is a fundamental probl...
research
04/02/2021

Multi-class motion-based semantic segmentation for ureteroscopy and laser lithotripsy

Kidney stones represent a considerable burden for public health-care sys...

Please sign up or login with your details

Forgot password? Click here to reset