Paracosm: A Language and Tool for Testing Autonomous Driving Systems

02/04/2019
by   Rupak Majumdar, et al.
0

Systematic testing of autonomous vehicles operating in complex real-world scenarios is a difficult and expensive problem. We present Paracosm, a reactive language for writing test scenarios for autonomous driving systems. Paracosm allows users to programmatically describe complex driving situations with specific visual features, e.g., road layout in an urban environment, as well as reactive temporal behaviors of cars and pedestrians. Paracosm programs are executed on top of a game engine that provides realistic physics simulation and visual rendering. The infrastructure allows systematic exploration of the state space, both for visual features (lighting, shadows, fog) and for reactive interactions with the environment (pedestrians, other traffic). We define a notion of test coverage for Paracosm configurations based on combinatorial testing and low dispersion sequences. Paracosm comes with an automatic test case generator that uses random sampling for discrete parameters and deterministic quasi-Monte Carlo generation for continuous parameters. Through an empirical evaluation, we demonstrate the modeling and testing capabilities of Paracosm on a suite of autonomous driving systems implemented using deep neural networks developed in research and education. We show how Paracosm can expose incorrect behaviors or degraded performance.

READ FULL TEXT

page 4

page 6

page 9

research
04/18/2018

Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components

Many organizations are developing autonomous driving systems, which are ...
research
08/28/2017

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

Recent advances in Deep Neural Networks (DNNs) have led to the developme...
research
08/10/2017

Systematic Testing of Convolutional Neural Networks for Autonomous Driving

We present a framework to systematically analyze convolutional neural ne...
research
09/13/2021

Neural Network Guided Evolutionary Fuzzing for Finding Traffic Violations of Autonomous Vehicles

Self-driving cars and trucks, autonomous vehicles (AVs), should not be a...
research
08/17/2021

An Empirical Testing of Autonomous Vehicle Simulator System for Urban Driving

Safety is one of the main challenges that prohibit autonomous vehicles (...
research
01/10/2023

Simulation-based Validation for Autonomous Driving Systems

Simulation is essential to validate autonomous driving systems. However,...
research
07/20/2023

Boundary State Generation for Testing and Improvement of Autonomous Driving Systems

Recent advances in Deep Neural Networks (DNNs) and sensor technologies a...

Please sign up or login with your details

Forgot password? Click here to reset