Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues

08/12/2017
by   Lu Chi, et al.
0

In recent years, autonomous driving algorithms using low-cost vehicle-mounted cameras have attracted increasing endeavors from both academia and industry. There are multiple fronts to these endeavors, including object detection on roads, 3-D reconstruction etc., but in this work we focus on a vision-based model that directly maps raw input images to steering angles using deep networks. This represents a nascent research topic in computer vision. The technical contributions of this work are three-fold. First, the model is learned and evaluated on real human driving videos that are time-synchronized with other vehicle sensors. This differs from many prior models trained from synthetic data in racing games. Second, state-of-the-art models, such as PilotNet, mostly predict the wheel angles independently on each video frame, which contradicts common understanding of driving as a stateful process. Instead, our proposed model strikes a combination of spatial and temporal cues, jointly investigating instantaneous monocular camera observations and vehicle's historical states. This is in practice accomplished by inserting carefully-designed recurrent units (e.g., LSTM and Conv-LSTM) at proper network layers. Third, to facilitate the interpretability of the learned model, we utilize a visual back-propagation scheme for discovering and visualizing image regions crucially influencing the final steering prediction. Our experimental study is based on about 6 hours of human driving data provided by Udacity. Comprehensive quantitative evaluations demonstrate the effectiveness and robustness of our model, even under scenarios like drastic lighting changes and abrupt turning. The comparison with other state-of-the-art models clearly reveals its superior performance in predicting the due wheel angle for a self-driving car.

READ FULL TEXT

page 2

page 4

page 6

page 7

page 8

page 9

page 11

research
09/29/2019

Vision-Based Autonomous Vehicle Control using the Two-Point Visual Driver Control Model

This work proposes a new self-driving framework that uses a human driver...
research
04/08/2019

Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks

A fundamental challenge in autonomous vehicles is adjusting the steering...
research
07/27/2021

Vision-Guided Forecasting – Visual Context for Multi-Horizon Time Series Forecasting

Autonomous driving gained huge traction in recent years, due to its pote...
research
03/17/2023

TBP-Former: Learning Temporal Bird's-Eye-View Pyramid for Joint Perception and Prediction in Vision-Centric Autonomous Driving

Vision-centric joint perception and prediction (PnP) has become an emerg...
research
12/04/2016

End-to-end Learning of Driving Models from Large-scale Video Datasets

Robust perception-action models should be learned from training data wit...
research
10/11/2018

ISA^2: Intelligent Speed Adaptation from Appearance

In this work we introduce a new problem named Intelligent Speed Adaptati...
research
12/11/2019

Self-Driving Car Steering Angle Prediction Based on Image Recognition

Self-driving vehicles have expanded dramatically over the last few years...

Please sign up or login with your details

Forgot password? Click here to reset