Searching for test data with feature diversity

09/18/2017
by   Robert Feldt, et al.
0

There is an implicit assumption in software testing that more diverse and varied test data is needed for effective testing and to achieve different types and levels of coverage. Generic approaches based on information theory to measure and thus, implicitly, to create diverse data have also been proposed. However, if the tester is able to identify features of the test data that are important for the particular domain or context in which the testing is being performed, the use of generic diversity measures such as this may not be sufficient nor efficient for creating test inputs that show diversity in terms of these features. Here we investigate different approaches to find data that are diverse according to a specific set of features, such as length, depth of recursion etc. Even though these features will be less general than measures based on information theory, their use may provide a tester with more direct control over the type of diversity that is present in the test data. Our experiments are carried out in the context of a general test data generation framework that can generate both numerical and highly structured data. We compare random sampling for feature-diversity to different approaches based on search and find a hill climbing search to be efficient. The experiments highlight many trade-offs that needs to be taken into account when searching for diversity. We argue that recurrent test data generation motivates building statistical models that can then help to more quickly achieve feature diversity.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/12/2011

A Factorial Experiment on Scalability of Search Based Software Testing

Software testing is an expensive process, which is vital in the industry...
research
08/27/2019

Towards Constraint Logic Programming over Strings for Test Data Generation

In order to properly test software, test data of a certain quality is ne...
research
10/19/2020

Using mutation testing to measure behavioural test diversity

Diversity has been proposed as a key criterion to improve testing effect...
research
11/29/2017

Enhancing Path-Oriented Test Data Generation Using Adaptive Random Testing Techniques

In this paper, we have developed an approach to generate test data for p...
research
02/01/2019

Practical Model-driven Data Generation for System Testing

The ability to generate test data is often a necessary prerequisite for ...
research
02/09/2022

Agree to Disagree: Diversity through Disagreement for Better Transferability

Gradient-based learning algorithms have an implicit simplicity bias whic...
research
05/21/2020

Unsupposable Test-data Generation for Machine-learned Software

As for software development by machine learning, a trained model is eval...

Please sign up or login with your details

Forgot password? Click here to reset