DeepAI
Log In Sign Up

Situation Coverage Testing for a Simulated Autonomous Car – an Initial Case Study

11/15/2019
by   Heather Hawkins, et al.
0

It is hard to test autonomous robot (AR) software because of the range and diversity of external situations (terrain, obstacles, humans, peer robots) that AR must deal with. Common measures of testing adequacy may not address this diversity. Explicit situation coverage has been proposed as a solution, but there has been little empirical study of its effectiveness. In this paper, we describe an implementation of situation coverage for testing a simple simulated autonomous road vehicle, and evaluate its ability to find seeded faults compared to a random test generation approach. In our experiments, the performance of the two methods is similar, with situation coverage having a very slight advantage. We conclude that situation coverage probably does not have a significant benefit over random generation for the type of simple, research-grade AR software used here. It will likely be valuable when applied to more complex and mature software.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/14/2019

There is Limited Correlation between Coverage and Robustness for Deep Neural Networks

Deep neural networks (DNN) are increasingly applied in safety-critical s...
07/09/2022

Testing of symmetry of innovations in autoregression

We consider a stationary linear AR(p) model with zero mean. The autoregr...
10/10/2020

Deep Neural Network Test Coverage: How Far Are We?

DNN testing is one of the most effective methods to guarantee the qualit...
04/05/2021

Conformal testing in a binary model situation

Conformal testing is a way of testing the IID assumption based on confor...
09/28/2022

Advising Autonomous Cars about the Rules of the Road

This paper describes (R)ules (o)f (T)he (R)oad (A)dvisor, an agent that ...
11/07/2017

Image Captioning and Classification of Dangerous Situations

Current robot platforms are being employed to collaborate with humans in...