NeurAll: Towards a Unified Model for Visual Perception in Automated Driving

02/10/2019
by   Ganesh Sistu, et al.
14

Convolutional Neural Networks (CNNs) are successfully used for the important automotive visual perception tasks including object recognition, motion and depth estimation, visual SLAM, etc. However, these tasks are independently explored and modeled. In this paper, we propose a joint multi-task network design called NeurAll for learning all tasks simultaneously. Our main motivation is the computational efficiency achieved by sharing the expensive initial convolutional layers between all tasks. Indeed, the main bottleneck in automated driving systems is the limited processing power available on deployment hardware. There could be other benefits in improving accuracy for some tasks and it eases development effort. It also offers scalability to add more tasks leveraging existing features and achieving better generalization. We survey various CNN based solutions for visual perception tasks in automated driving. Then we propose a unified CNN model for the important tasks and discuss several advanced optimization and architecture design techniques to improve the baseline model. The paper is partly review and partly positional with demonstration of several preliminary results promising for future research. Firstly, we show that an efficient two-task model performing semantic segmentation and object detection achieves similar accuracies compared to separate models on various datasets with minimized runtime. We then illustrate that using depth regression as auxiliary task improves semantic segmentation and using multi-stream semantic segmentation outperforms one-stream semantic segmentation. The two-task network achieves 30 fps on an automotive grade low power SOC for 1280x384 image resolution

READ FULL TEXT

page 1

page 2

page 5

page 6

research
01/12/2019

Real-time Joint Object Detection and Semantic Segmentation Network for Automated Driving

Convolutional Neural Networks (CNN) are successfully used for various vi...
research
01/17/2019

AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving

Decision making in automated driving is highly specific to the environme...
research
06/11/2022

Surround-View Cameras based Holistic Visual Perception for Automated Driving

The formation of eyes led to the big bang of evolution. The dynamics cha...
research
02/04/2020

Selective Segmentation Networks Using Top-Down Attention

Convolutional neural networks model the transformation of the input sens...
research
01/19/2021

Multi-Task Network Pruning and Embedded Optimization for Real-time Deployment in ADAS

Camera-based Deep Learning algorithms are increasingly needed for percep...
research
11/25/2021

Robust Object Detection with Multi-input Multi-output Faster R-CNN

Recent years have seen impressive progress in visual recognition on many...
research
04/11/2019

YUVMultiNet: Real-time YUV multi-task CNN for autonomous driving

In this paper, we propose a multi-task convolutional neural network (CNN...

Please sign up or login with your details

Forgot password? Click here to reset