Learning How to Search: Generating Effective Test Cases Through Adaptive Fitness Function Selection

by   Hussein Almulla, et al.

Search-based test generation is guided by feedback from one or more fitness functions - scoring functions that judge solution optimality. Choosing informative fitness functions is crucial to meeting the goals of a tester. Unfortunately, many goals - such as forcing the class-under-test to throw exceptions, increasing test suite diversity, and attaining Strong Mutation Coverage - do not have effective fitness function formulations. We propose that meeting such goals requires treating fitness function identification as a secondary optimization step. An adaptive algorithm that can vary the selection of fitness functions could adjust its selection throughout the generation process to maximize goal attainment, based on the current population of test suites. To test this hypothesis, we have implemented two reinforcement learning algorithms in the EvoSuite unit test generation framework, and used these algorithms to dynamically set the fitness functions used during generation for the three goals identified above. We have evaluated our framework, EvoSuiteFIT, on a set of real Java case examples. EvoSuiteFIT techniques attain significant improvements for two of the three goals, and show small improvements on the third when the number of generations of evolution is fixed. Additionally, for all goals, EvoSuiteFIT detects faults missed by the other techniques. The ability to adjust fitness functions allows EvoSuiteFIT to make strategic choices that efficiently produce more effective test suites, and examining its choices offers insight into how to attain our testing goals. We find that AFFS is a powerful technique to apply when an effective fitness function does not already exist for generating tests to achieve a testing goal.


page 1

page 2

page 3

page 4


Basic Block Coverage for Search-based Unit Testing and Crash Reproduction

Search-based techniques have been widely used for white-box test generat...

Selectively Combining Multiple Coverage Goals in Search-Based Unit Test Generation

Unit testing is a critical part of software development process, ensurin...

ASBSO: An Improved Brain Storm Optimization With Flexible Search Length and Memory-Based Selection

Brain storm optimization (BSO) is a newly proposed population-based opti...

Szenario-Optimierung für die Absicherung von automatisierten und autonomen Fahrsystemen

The verification and validation of automated and autonomous driving syst...

Bet and Run for Test Case Generation

Anyone working in the technology sector is probably familiar with the qu...

Epsilon-Lexicase Selection for Regression

Lexicase selection is a parent selection method that considers test case...

A Comprehensive Empirical Evaluation of Generating Test Suites for Mobile Applications with Diversity

Context: In search-based software engineering we often use popular heuri...