Why are Some Bugs Non-Reproducible? An Empirical Investigation using Data Fusion

08/11/2021
by   Mohammad Masudur Rahman, et al.
0

Software developers attempt to reproduce software bugs to understand their erroneous behaviours and to fix them. Unfortunately, they often fail to reproduce (or fix) them, which leads to faulty, unreliable software systems. However, to date, only a little research has been done to better understand what makes the software bugs non-reproducible. In this paper, we conduct a multimodal study to better understand the non-reproducibility of software bugs. First, we perform an empirical study using 576 non-reproducible bug reports from two popular software systems (Firefox, Eclipse) and identify 11 key factors that might lead a reported bug to non-reproducibility. Second, we conduct a user study involving 13 professional developers where we investigate how the developers cope with non-reproducible bugs. We found that they either close these bugs or solicit for further information, which involves long deliberations and counter-productive manual searches. Third, we offer several actionable insights on how to avoid non-reproducibility (e.g., false-positive bug report detector) and improve reproducibility of the reported bugs (e.g., sandbox for bug reproduction) by combining our analyses from multiple studies (e.g., empirical study, developer study).

READ FULL TEXT

page 6

page 8

research
01/28/2023

Characterizing and Detecting WebAssembly Runtime Bugs

WebAssembly (abbreviated WASM) has emerged as a promising language of th...
research
06/24/2022

Bugs in Machine Learning-based Systems: A Faultload Benchmark

The rapid escalation of applying Machine Learning (ML) in various domain...
research
10/07/2022

Understanding and Supporting Debugging Workflows in Multiverse Analysis

Multiverse analysis-a paradigm for statistical analysis that considers a...
research
03/17/2022

An Empirical Study of Bugs in Eclipse Stable Internal Interfaces

The Eclipse framework is a popular and widely used framework that has be...
research
12/19/2021

Reproducibility Challenges and Their Impacts on Technical Q A Websites: The Practitioners' Perspectives

Software developers often look for solutions to their code-level problem...
research
06/17/2019

Assessing the Quality of the Steps to Reproduce in Bug Reports

A major problem with user-written bug reports, indicated by developers a...
research
09/09/2021

The challenge of reproducible ML: an empirical study on the impact of bugs

Reproducibility is a crucial requirement in scientific research. When re...

Please sign up or login with your details

Forgot password? Click here to reset