Breakdown of a Benchmark Score Without Internal Analysis of Benchmarking Program

10/20/2016
by   Naoki Matagawa, et al.
0

A breakdown of a benchmark score is how much each aspect of the system performance affects the score. Existing methods require internal analysis on the benchmarking program and then involve the following problems: (1) require a certain amount of labor for code analysis, profiling, simulation, and so on and (2) require the benchmarking program itself. In this paper, we present a method for breaking down a benchmark score without internal analysis of the benchmarking program. The method utilizes regression analysis of benchmark scores on a number of systems. Experimental results with 3 benchmarks on 15 Android smartphones showed that our method could break down those benchmark scores even though there is room for improvement in accuracy.

READ FULL TEXT
research
08/25/2021

Method Calls Frequency-Based Tie-Breaking Strategy For Software Fault Localization

In Spectrum-Based Fault Localization (SBFL), a suspiciousness score is a...
research
04/01/2019

A benchmark for C program verification

We present twenty-five C programs, as a benchmark for C program verifica...
research
05/17/2023

Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks

The evaluation of natural language processing (NLP) systems is crucial f...
research
07/27/2023

Benchmarking Performance of Deep Learning Model for Material Segmentation on Two HPC Systems

Performance Benchmarking of HPC systems is an ongoing effort that seeks ...
research
06/10/2021

PSB2: The Second Program Synthesis Benchmark Suite

For the past six years, researchers in genetic programming and other pro...
research
05/31/2015

OR-Benchmark: An Open and Reconfigurable Digital Watermarking Benchmarking Framework

Benchmarking digital watermarking algorithms is not an easy task because...
research
03/23/2022

A Computationally Efficient Approach to Fully Bayesian Benchmarking

In small area estimation, it is often necessary to resort to model-based...

Please sign up or login with your details

Forgot password? Click here to reset