An investigation into the state-of-the-practice autonomous driving testing

by   Guannan Lou, et al.

Autonomous driving shows great potential to reform modern transportation and its safety is attracting much attention from public. Autonomous driving systems generally include deep neural networks (DNNs) for gaining better performance (e.g., accuracy on object detection and trajectory prediction). However, compared with traditional software systems, this new paradigm (i.e., program + DNNs) makes software testing more difficult. Recently, software engineering community spent significant effort in developing new testing methods for autonomous driving systems. However, it is not clear that what extent those testing methods have addressed the needs of industrial practitioners of autonomous driving. To fill this gap, in this paper, we present the first comprehensive study to identify the current practices and needs of testing autonomous driving systems in industry. We conducted semi-structured interviews with developers from 10 autonomous driving companies and surveyed 100 developers who have worked on autonomous driving systems. Through thematic analysis of interview and questionnaire data, we identified five urgent needs of testing autonomous driving systems from industry. We further analyzed the limitations of existing testing methods to address those needs and proposed several future directions for software testing researchers.


page 4

page 7

page 8

page 9

page 10

page 11


Industry Practices for Challenging Autonomous Driving Systems with Critical Scenarios

Testing autonomous driving systems for safety and reliability is extreme...

Challenges in Designing Datasets and Validation for Autonomous Driving

Autonomous driving is getting a lot of attention in the last decade and ...

Rethinking Certification for Higher Trust and Ethical Safeguarding of Autonomous Systems

With the increasing complexity of software permeating critical domains s...

Future Vision of Dynamic Certification Schemes for Autonomous Systems

As software becomes increasingly pervasive in critical domains like auto...

Reducing DNN Labelling Cost using Surprise Adequacy: An Industrial Case Study for Autonomous Driving

Deep Neural Networks (DNNs) are rapidly being adopted by the automotive ...

A Comprehensive Review on Ontologies for Scenario-based Testing in the Context of Autonomous Driving

The verification and validation of autonomous driving vehicles remains a...

Energy-Efficient Adaptive System Reconfiguration for Dynamic Deadlines in Autonomous Driving

The increasing computing demands of autonomous driving applications make...

Please sign up or login with your details

Forgot password? Click here to reset