Putting Them under Microscope: A Fine-Grained Approach for Detecting Redundant Test Cases in Natural Language

10/04/2022
by   Zhiyuan Chang, et al.
0

Natural language (NL) documentation is the bridge between software managers and testers, and NL test cases are prevalent in system-level testing and other quality assurance activities. Due to reasons such as requirements redundancy, parallel testing, and tester turnover within long evolving history, there are inevitably lots of redundant test cases, which significantly increase the cost. Previous redundancy detection approaches typically treat the textual descriptions as a whole to compare their similarity and suffer from low precision. Our observation reveals that a test case can have explicit test-oriented entities, such as tested function Components, Constraints, etc; and there are also specific relations between these entities. This inspires us with a potential opportunity for accurate redundancy detection. In this paper, we first define five test-oriented entity categories and four associated relation categories and re-formulate the NL test case redundancy detection problem as the comparison of detailed testing content guided by the test-oriented entities and relations. Following that, we propose Tscope, a fine-grained approach for redundant NL test case detection by dissecting test cases into atomic test tuple(s) with the entities restricted by associated relations. To serve as the test case dissection, Tscope designs a context-aware model for the automatic entity and relation extraction. Evaluation on 3,467 test cases from ten projects shows Tscope could achieve 91.8 recall, and 82.4 and commonly-used classifiers. This new formulation of the NL test case redundant detection problem can motivate the follow-up studies to further improve this task and other related tasks involving NL descriptions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/14/2021

Identifying Similar Test Cases That Are Specified in Natural Language

Software testing is still a manual process in many industries, despite t...
research
05/13/2022

AEON: A Method for Automatic Evaluation of NLP Test Cases

Due to the labor-intensive nature of manual test oracle construction, va...
research
07/21/2021

Fine-Grained Causality Extraction From Natural Language Requirements Using Recursive Neural Tensor Networks

[Context:] Causal relations (e.g., If A, then B) are prevalent in functi...
research
10/28/2022

ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolutionary Search

Executing large test suites is time and resource consuming, sometimes im...
research
08/22/2019

Automated Generation of Test Models from Semi-Structured Requirements

[Context:] Model-based testing is an instrument for automated generation...
research
07/15/2018

Visualizing test diversity to support test optimisation

Diversity has been used as an effective criteria to optimise test suites...
research
07/13/2023

Crucible: Graphical Test Cases for Alloy Models

Alloy is a declarative modeling language that is well suited for verifyi...

Please sign up or login with your details

Forgot password? Click here to reset