DeepAI AI Chat
Log In Sign Up

Multi-stream CNN based Video Semantic Segmentation for Automated Driving

by   Ganesh Sistu, et al.

Majority of semantic segmentation algorithms operate on a single frame even in the case of videos. In this work, the goal is to exploit temporal information within the algorithm model for leveraging motion cues and temporal consistency. We propose two simple high-level architectures based on Recurrent FCN (RFCN) and Multi-Stream FCN (MSFCN) networks. In case of RFCN, a recurrent network namely LSTM is inserted between the encoder and decoder. MSFCN combines the encoders of different frames into a fused encoder via 1x1 channel-wise convolution. We use a ResNet50 network as the baseline encoder and construct three networks namely MSFCN of order 2 & 3 and RFCN of order 2. MSFCN-3 produces the best results with an accuracy improvement of 9 Highway and New York-like city scenarios in the SYNTHIA-CVPR'16 dataset using mean IoU metric. MSFCN-3 also produced 11 datasets over the baseline FCN network. We also designed an efficient version of MSFCN-2 and RFCN-2 using weight sharing among the two encoders. The efficient MSFCN-2 provided an improvement of 11 with negligible increase in computational complexity compared to the baseline version.


page 3

page 6

page 7


Semantic Segmentation of Video Sequences with Convolutional LSTMs

Most of the semantic segmentation approaches have been developed for sin...

UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information

Semantic segmentation of aerial videos has been extensively used for dec...

Design of Real-time Semantic Segmentation Decoder for Automated Driving

Semantic segmentation remains a computationally intensive algorithm for ...

Using Image Priors to Improve Scene Understanding

Semantic segmentation algorithms that can robustly segment objects acros...

Video Time: Properties, Encoders and Evaluation

Time-aware encoding of frame sequences in a video is a fundamental probl...

Multi-class motion-based semantic segmentation for ureteroscopy and laser lithotripsy

Kidney stones represent a considerable burden for public health-care sys...