Log In Sign Up

Attention Please: Consider Mockito when Evaluating Newly Released Automated Program Repair Techniques

by   Shangwen Wang, et al.

Automated program repair (APR) has caused widespread concern in recent years with many techniques being proposed. Meanwhile, a number of benchmarks have been established for evaluating the performances of APR tools and Defects4J is a wildly used database among them. However, bugs in a later released project, Mockito, do not receive much attention in recent evaluations. In this paper, we aim to figure out whether we should consider Mockito bugs when evaluating APR tools. Our findings show that: 1) Mockito bugs are not more complex for repairing compared with bugs from other projects; 2) state-of-the-art tools have fixed some bugs with the same repair patterns as Mockito bugs; but 3) state-of-the-art tools possess poor performances on Mockito bugs (Nopol can only correctly fix one bug while SimFix and CapGen cannot fix any bug in Mockito even if all the buggy points are exposed). We conclude from the results that the breakthrough in this project may represent a huge improvement in repairing abilities and we should consider Mockito when evaluating newly released APR tools. We further find out a unique repair action existing in Mockito bugs named external package introduction. This may shed light on future researches about APR techniques.


page 1

page 4


Attention Please: Consider Mockito when Evaluating Newly Proposed Automated Program Repair Techniques

Automated program repair (APR) has attracted widespread attention in rec...

Towards Extending the Range of Bugs That Automated Program Repair Can Handle

Modern automated program repair (APR) is well-tuned to finding and repai...

Improving Automated Program Repair with Domain Adaptation

Automated Program Repair (APR) is defined as the process of fixing a bug...

Towards More Reliable Automated Program Repair by Integrating Static Analysis Techniques

A long-standing open challenge for automated program repair is the overf...

Sub-Turing Islands in the Wild

Recently, there has been growing debate as to whether or not static anal...

DroidBugs: An Android Benchmark for Automated Program Repair

Automated Program Repair (APR) is an emerging research field. Many APR t...

Understanding and Supporting Debugging Workflows in Multiverse Analysis

Multiverse analysis-a paradigm for statistical analysis that considers a...