Bears: An Extensible Java Bug Benchmark for Automatic Program Repair Studies

01/17/2019
by   Fernanda Madeiral, et al.
0

Benchmarks of bugs are essential to empirically evaluate automatic program repair tools. In this paper, we present Bears, a project for collecting and storing bugs into an extensible bug benchmark for automatic repair studies in Java. The collection of bugs relies on commit building state from Continuous Integration (CI) to find potential pairs of buggy and patched program versions from open-source projects hosted on GitHub. Each pair of program versions passes through a pipeline where an attempt of reproducing a bug and its patch is performed. The core step of the reproduction pipeline is the execution of the test suite of the program on both program versions. If a test failure is found in the buggy program version candidate and no test failure is found in its patched program version candidate, a bug and its patch were successfully reproduced. The uniqueness of Bears is the usage of CI (builds) to identify buggy and patched program version candidates, which has been widely adopted in the last years in open-source projects. This approach allows us to collect bugs from a diversity of projects beyond mature projects that use bug tracking systems. Moreover, Bears was designed to be publicly available and to be easily extensible by the research community through automatic creation of branches with bugs in a given GitHub repository, which can be used for pull requests in the Bears repository. We present in this paper the approach employed by Bears, and we deliver the version 1.0 of Bears, which contains 251 reproducible bugs collected from 72 projects that use the Travis CI and Maven build environment.

READ FULL TEXT
research
05/30/2019

How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Dataset

Program repair is an important but difficult software engineering proble...
research
11/10/2018

Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs

We propose NOPOL, an approach to automatic repair of buggy conditional s...
research
04/20/2023

Finding Bug-Inducing Program Environments

Some bugs cannot be exposed by program inputs, but only by certain progr...
research
07/16/2021

Towards a Benchmark Set for Program Repair Based on Partial Fixes

Software bugs significantly contribute to software cost and increase the...
research
12/16/2019

Human-In-The-Loop Automatic Program Repair

We introduce Learn2fix, the first human-in-the-loop, semi-automatic repa...
research
02/07/2019

How Different Are Different diff Algorithms in Git? Use --histogram for Code Changes

Automatic identification of the differences between two versions of a fi...

Please sign up or login with your details

Forgot password? Click here to reset