Benchmarking of Java Verification Tools at the Software Verification Competition (SV-COMP)

09/11/2018
by   Lucas Cordeiro, et al.
0

Empirical evaluation of verification tools by benchmarking is a common method in software verification research. The Competition on Software Verification (SV-COMP) aims at standardization and reproducibility of benchmarking within the software verification community on an annual basis, through comparative evaluation of fully automatic software verifiers for C programs. Building upon this success, here we describe our proposal to re-use the ecosystem developed around SV-COMP for benchmarking Java verification tools. We provide a detailed description of the rules for benchmark verification tasks, the integration of new tools into SV-COMP's benchmarking framework and also give experimental results of a benchmarking run on three state-of-the-art Java verification tools, JPF-SE, JayHorn and JBMC.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/11/2023

Reproduction Report for SV-COMP 2023

The Competition on Software Verification (SV-COMP) is a large computatio...
research
07/04/2020

Building benchmarking frameworks for supporting replicability and reproducibility: spatial and textual analysis as an example

Replicability and reproducibility (R R) are critical for the long-term...
research
11/16/2017

Software Metric Framework

Many researchers have criticized the field of Software Complexity metric...
research
08/11/2020

Cooperative Verification via Collective Invariant Generation

Software verification has recently made enormous progress due to the dev...
research
09/16/2023

Btor2MLIR: A Format and Toolchain for Hardware Verification

Formats for representing and manipulating verification problems are extr...
research
08/15/2016

Robust benchmarking in noisy environments

We propose a benchmarking strategy that is robust in the presence of tim...
research
05/21/2019

Verification Artifacts in Cooperative Verification: Survey and Unifying Component Framework

The goal of cooperative verification is to combine verification approach...

Please sign up or login with your details

Forgot password? Click here to reset