DeepAI AI Chat
Log In Sign Up

Testing Causality in Scientific Modelling Software

by   Andrew G. Clark, et al.
The University of Sheffield
Universität Ulm

From simulating galaxy formation to viral transmission in a pandemic, scientific models play a pivotal role in developing scientific theories and supporting government policy decisions that affect us all. Given these critical applications, a poor modelling assumption or bug could have far-reaching consequences. However, scientific models possess several properties that make them notoriously difficult to test, including a complex input space, long execution times, and non-determinism, rendering existing testing techniques impractical. In fields such as epidemiology, where researchers seek answers to challenging causal questions, a statistical methodology known as Causal Inference has addressed similar problems, enabling the inference of causal conclusions from noisy, biased, and sparse data instead of costly experiments. This paper introduces the Causal Testing Framework: a framework that uses Causal Inference techniques to establish causal effects from existing data, enabling users to conduct software testing activities concerning the effect of a change, such as Metamorphic Testing and Sensitivity Analysis, a posteriori. We present three case studies covering real-world scientific models, demonstrating how the Causal Testing Framework can infer test outcomes from reused, confounded test data to provide an efficient solution for testing scientific modelling software.


page 1

page 2

page 3

page 4


Bayesian causal inference in automotive software engineering and online evaluation

Randomised field experiments, such as A/B testing, have long been the go...

Applying causal inference to inform early-childhood policy from administrative data

Improving public policy is one of the key roles of governments, and they...

Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition

Statisticians have made great strides towards assumption-free estimation...

A review of spatial causal inference methods for environmental and epidemiological applications

The scientific rigor and computational methods of causal inference have ...

Network Data

Many economic activities are embedded in networks: sets of agents and th...

A general framework for causal classification

In many applications, there is a need to predict the effect of an interv...

Reasoning-Based Software Testing

With software systems becoming increasingly pervasive and autonomous, ou...