Log In Sign Up

Situation Coverage Testing for a Simulated Autonomous Car – an Initial Case Study

by   Heather Hawkins, et al.

It is hard to test autonomous robot (AR) software because of the range and diversity of external situations (terrain, obstacles, humans, peer robots) that AR must deal with. Common measures of testing adequacy may not address this diversity. Explicit situation coverage has been proposed as a solution, but there has been little empirical study of its effectiveness. In this paper, we describe an implementation of situation coverage for testing a simple simulated autonomous road vehicle, and evaluate its ability to find seeded faults compared to a random test generation approach. In our experiments, the performance of the two methods is similar, with situation coverage having a very slight advantage. We conclude that situation coverage probably does not have a significant benefit over random generation for the type of simple, research-grade AR software used here. It will likely be valuable when applied to more complex and mature software.


page 1

page 2

page 3

page 4


There is Limited Correlation between Coverage and Robustness for Deep Neural Networks

Deep neural networks (DNN) are increasingly applied in safety-critical s...

Testing of symmetry of innovations in autoregression

We consider a stationary linear AR(p) model with zero mean. The autoregr...

Deep Neural Network Test Coverage: How Far Are We?

DNN testing is one of the most effective methods to guarantee the qualit...

Conformal testing in a binary model situation

Conformal testing is a way of testing the IID assumption based on confor...

Advising Autonomous Cars about the Rules of the Road

This paper describes (R)ules (o)f (T)he (R)oad (A)dvisor, an agent that ...

Image Captioning and Classification of Dangerous Situations

Current robot platforms are being employed to collaborate with humans in...