Reducing the Effort for Systematic Reviews in Software Engineering

08/19/2019
by   Francesco Osborne, et al.
0

Context. Systematic Reviews (SRs) are means for collecting and synthesizing evidence from the identification and analysis of relevant studies from multiple sources. To this aim, they use a well-defined methodology meant to mitigate the risks of biases and ensure repeatability for later updates. SRs, however, involve significant effort. Goal. The goal of this paper is to introduce a novel methodology that reduces the amount of manual tedious tasks involved in SRs while taking advantage of the value provided by human expertise. Method. Starting from current methodologies for SRs, we replaced the steps of keywording and data extraction with an automatic methodology for generating a domain ontology and classifying the primary studies. This methodology has been applied in the Software Engineering sub-area of Software Architecture and evaluated by human annotators. Results. The result is a novel Expert-Driven Automatic Methodology, EDAM, for assisting researchers in performing SRs. EDAM combines ontology-learning techniques and semantic technologies with the human-in-the-loop. The first (thanks to automation) fosters scalability, objectivity, reproducibility and granularity of the studies; the second allows tailoring to the specific focus of the study at hand and knowledge reuse from domain experts. We evaluated EDAM on the field of Software Architecture against six senior researchers. As a result, we found that the performance of the senior researchers in classifying papers was not statistically significantly different from EDAM. Conclusions. Thanks to automation of the less-creative steps in SRs, our methodology allows researchers to skip the tedious tasks of keywording and manually classifying primary studies, thus freeing effort for the analysis and the discussion.

READ FULL TEXT
research
04/05/2023

Architectural Support for Software Performance in Continuous Software Engineering: A Systematic Mapping Study

The continuous software engineering paradigm is gaining popularity in mo...
research
06/09/2020

Guidelines for the Search Strategy to Update Systematic Literature Reviews in Software Engineering

Context: Systematic Literature Reviews (SLRs) have been adopted within S...
research
08/31/2021

Towards Sustainability of Systematic Literature Reviews

Background: The software engineering community has increasingly conducte...
research
03/02/2021

Stop Building Castles on a Swamp! The Crisis of Reproducing Automatic Search in Evidence-based Software Engineering

The evidence-based approach has increasingly been employed to synthesize...
research
06/26/2019

Software Engineering Research Community Viewpoints on Rapid Reviews

Background: One of the most important current challenges of Software Eng...
research
08/12/2019

Methodological Issues in Observational Studies

Background. Starting from the 1960s, practitioners and researchers have ...
research
05/21/2020

Systematic Literature Reviews in Software Engineering – Enhancement of the Study Selection Process using Cohen's Kappa Statistic

Context: Systematic literature reviews (SLRs) rely on a rigorous and aud...

Please sign up or login with your details

Forgot password? Click here to reset