DeepAI AI Chat
Log In Sign Up

HyperPUT: Generating Synthetic Faulty Programs to Challenge Bug-Finding Tools

by   Riccardo Felici, et al.

As research in automatically detecting bugs grows and produces new techniques, having suitable collections of programs with known bugs becomes crucial to reliably and meaningfully compare the effectiveness of these techniques. Most of the existing approaches rely on benchmarks collecting manually curated real-world bugs, or synthetic bugs seeded into real-world programs. Using real-world programs entails that extending the existing benchmarks or creating new ones remains a complex time-consuming task. In this paper, we propose a complementary approach that automatically generates programs with seeded bugs. Our technique, called HyperPUT, builds C programs from a "seed" bug by incrementally applying program transformations (introducing programming constructs such as conditionals, loops, etc.) until a program of the desired size is generated. In our experimental evaluation, we demonstrate how HyperPUT can generate buggy programs that can challenge in different ways the capabilities of modern bug-finding tools, and some of whose characteristics are comparable to those of bugs in existing benchmarks. These results suggest that HyperPUT can be a useful tool to support further research in bug-finding techniques – in particular their empirical evaluations.


page 1

page 2

page 3

page 4


Effects of Hints on Debugging Scratch Programs: An Empirical Study with Primary School Teachers in Training

Bugs in learners' programs are often the result of fundamental misconcep...

Automated Customized Bug-Benchmark Generation

We introduce Bug-Injector, a system that automatically creates benchmark...

Evaluating Synthetic Bugs

Fuzz testing has been used to find bugs in programs since the 1990s, but...

Finding Bugs with Specification-Based Testing is Easy!

Automated specification-based testing has a long history with several no...

Bonsai: Synthesis-Based Reasoning for Type Systems

We describe algorithms for symbolic reasoning about executable models of...

FuzzerAid: Grouping Fuzzed Crashes Based On Fault Signatures

Fuzzing has been an important approach for finding bugs and vulnerabilit...

ReduKtor: How We Stopped Worrying About Bugs in Kotlin Compiler

Bug localization is well-known to be a difficult problem in software eng...