Visual GUI testing in practice: An extended industrial case study

05/19/2020
by   Vahid Garousi, et al.
0

Context: Visual GUI testing (VGT) is referred to as the latest generation GUI-based testing. It is a tool-driven technique, which uses image recognition for interacting with and asserting the behavior of the system under test. Motivated by the industrial need of a large Turkish software and systems company providing solutions in the areas of defense and IT sector, an action-research project was recently initiated to implement VGT in several teams and projects in the company. Objective: To address the above needs, we planned and carried out an empirical investigation with the goal of assessing VGT using two tools (Sikuli and JAutomate). The purpose was to determine a suitable approach and tool for VGT of a given project (software product) in the company, increase the know-how in the company's test teams. Method: Using an action-research case-study design, we investigated the use of VGT in the studied organization. Specifically, using the two selected VGT tools, we conducted a quantitative and a qualitative evaluation of VGT. Results: By assessing the list of Challenges, Problems and Limitations (CPL), proposed in previous work, in the context of our empirical study, we found that test-tool- and SUT-related CPLs were quite comparable to a previous empirical study, e.g., the synchronization between SUT and test tools were not always robust and there were failures in test tools' image recognition features. When assessing the types of test maintenance activities, when executing the automated test cases on next versions of the SUTs, for the case of the two test tools, we found that about half of the test cases (59.1 the next version. Conclusion: By our results, we confirm some of the previously-reported issues when conducting VGT. Further, we highlight some additional challenges in test maintenance when using VGT.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 11

page 16

page 22

02/28/2021

On Introducing Automatic Test Case Generation in Practice: A Success Story and Lessons Learned

The level and quality of automation dramatically affects software testin...
05/26/2020

Assessing the maturity of software testing services using CMMI-SVC: An industrial case study

Context: While many companies conduct their software testing activities ...
04/24/2018

Transferring Interactive Search-Based Software Testing to Industry

Search-Based Software Testing (SBST) is the application of optimization ...
07/08/2019

Estimating Return on Investment for GUI Test Automation Tools

Automated graphical user interface (GUI) tests can reduce manual testing...
01/22/2020

CDST: A Toolkit for Testing Cockpit Display Systems of Avionics

Avionics are highly critical systems that require extensive testing gove...
04/05/2021

Model-based testing in practice: An experience report from the web applications domain

In the context of a large software testing company, we have deployed the...
07/15/2018

Visualizing test diversity to support test optimisation

Diversity has been used as an effective criteria to optimise test suites...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.