Automated Patch Assessment for Program Repair at Scale

09/30/2019
by   He Ye, et al.
0

In this paper, we do automatic correctness assessment for patches generated by program repair techniques. We consider the human patch as ground truth oracle and randomly generate tests based on it, i.e., Random testing with Ground Truth – RGT. We build a curated dataset of 638 patches for Defects4J generated by 14 state-of-the-art repair systems. We evaluate automated patch assessment on our dataset which is, to our knowledge, the largest ever. The results of this study are novel and significant. First, we show that 10 patches from previous research classified as correct by their respective authors are actually overfitting. Second, we demonstrate that the human patch is not the perfect ground truth. Third, we precisely measure the trade-off between the time spent for test generation and the benefits for automated patch assessment at scale.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/09/2018

A Comprehensive Study of Automatic Program Repair on the QuixBugs Benchmark

Automatic program repair papers tend to repeatedly use the same benchmar...
research
08/07/2020

Evaluating Representation Learning of Code Changes for Predicting Patch Correctness in Program Repair

A large body of the literature of automated program repair develops appr...
research
05/15/2018

On Reliability of Patch Correctness Assessment

Current state-of-the-art automatic software repair (ASR) techniques rely...
research
12/08/2018

Production-Driven Patch Generation

We present an original concept for patch generation: we propose to do it...
research
10/26/2019

Automated Classification of Overfitting Patches with Statically Extracted Code Features

Automatic program repair (APR) aims to reduce the cost of manually fixin...
research
08/28/2018

CFAAR: Control Flow Alteration to Assist Repair

We present CFAAR, a program repair assistance technique that operates by...
research
03/25/2020

Patch Quality and Diversity of Invariant-Guided Search-Based Program Repair

Most automatic program repair techniques rely on test cases to specify c...

Please sign up or login with your details

Forgot password? Click here to reset