Log In Sign Up

Visual GUI testing in practice: An extended industrial case study

by   Vahid Garousi, et al.

Context: Visual GUI testing (VGT) is referred to as the latest generation GUI-based testing. It is a tool-driven technique, which uses image recognition for interacting with and asserting the behavior of the system under test. Motivated by the industrial need of a large Turkish software and systems company providing solutions in the areas of defense and IT sector, an action-research project was recently initiated to implement VGT in several teams and projects in the company. Objective: To address the above needs, we planned and carried out an empirical investigation with the goal of assessing VGT using two tools (Sikuli and JAutomate). The purpose was to determine a suitable approach and tool for VGT of a given project (software product) in the company, increase the know-how in the company's test teams. Method: Using an action-research case-study design, we investigated the use of VGT in the studied organization. Specifically, using the two selected VGT tools, we conducted a quantitative and a qualitative evaluation of VGT. Results: By assessing the list of Challenges, Problems and Limitations (CPL), proposed in previous work, in the context of our empirical study, we found that test-tool- and SUT-related CPLs were quite comparable to a previous empirical study, e.g., the synchronization between SUT and test tools were not always robust and there were failures in test tools' image recognition features. When assessing the types of test maintenance activities, when executing the automated test cases on next versions of the SUTs, for the case of the two test tools, we found that about half of the test cases (59.1 the next version. Conclusion: By our results, we confirm some of the previously-reported issues when conducting VGT. Further, we highlight some additional challenges in test maintenance when using VGT.


page 11

page 16

page 22


On Introducing Automatic Test Case Generation in Practice: A Success Story and Lessons Learned

The level and quality of automation dramatically affects software testin...

Assessing the maturity of software testing services using CMMI-SVC: An industrial case study

Context: While many companies conduct their software testing activities ...

Transferring Interactive Search-Based Software Testing to Industry

Search-Based Software Testing (SBST) is the application of optimization ...

Estimating Return on Investment for GUI Test Automation Tools

Automated graphical user interface (GUI) tests can reduce manual testing...

When Traceability Goes Awry: an Industrial Experience Report

The concept of traceability between artifacts is considered an enabler f...

Towards automated open source assessment – An empirical study

The open source software (OSS) assessment has become important given the...

An Empirical Study on Technical Debt in a Finnish SME

Objective. In this work, we report the experience of a Finnish SME in ma...