On Introducing Automatic Test Case Generation in Practice: A Success Story and Lessons Learned

by   Matteo Brunetto, et al.

The level and quality of automation dramatically affects software testing activities, determines costs and effectiveness of the testing process, and largely impacts on the quality of the final product. While costs and benefits of automating many testing activities in industrial practice (including managing the quality process, executing large test suites, and managing regression test suites) are well understood and documented, the benefits and obstacles of automatically generating system test suites in industrial practice are not well reported yet, despite the recent progresses of automated test case generation tools. Proprietary tools for automatically generating test cases are becoming common practice in large software organisations, and commercial tools are becoming available for some application domains and testing levels. However, generating system test cases in small and medium-size software companies is still largely a manual, inefficient and ad-hoc activity. This paper reports our experience in introducing techniques for automatically generating system test suites in a medium-size company. We describe the technical and organisational obstacles that we faced when introducing automatic test case generation in the development process of the company, and present the solutions that we successfully experienced in that context. In particular, the paper discusses the problems of automating the generation of test cases by referring to a customised ERP application that the medium-size company developed for a third party multinational company, and presents ABT2.0, the test case generator that we developed by tailoring ABT, a research state-of-the-art GUI test generator, to their industrial environment. This paper presents the new features of ABT2.0, and discusses how these new features address the issues that we faced.


page 1

page 2

page 3

page 4


Visual GUI testing in practice: An extended industrial case study

Context: Visual GUI testing (VGT) is referred to as the latest generatio...

An Empirical Study on Technical Debt in a Finnish SME

Objective. In this work, we report the experience of a Finnish SME in ma...

When Traceability Goes Awry: an Industrial Experience Report

The concept of traceability between artifacts is considered an enabler f...

Model-based testing in practice: An experience report from the web applications domain

In the context of a large software testing company, we have deployed the...

Estimating Return on Investment for GUI Test Automation Tools

Automated graphical user interface (GUI) tests can reduce manual testing...

Compositional Test Generation of Industrial Synchronous Systems

Synchronous systems provide a basic model of embedded systems and indust...

Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD

Test-Driven Development (TDD) has been claimed to increase external soft...