An Efficient Method for Uncertainty Propagation in Robust Software Performance Estimation

01/15/2018
by   Aldeida Aleti, et al.
0

Software engineers often have to estimate the performance of a software system before having full knowledge of the system parameters, such as workload and operational profile. These uncertain parameters inevitably affect the accuracy of quality evaluations, and the ability to judge if the system can continue to fulfil performance requirements if parameter results are different from expected. Previous work has addressed this problem by modelling the potential values of uncertain parameters as probability distribution functions, and estimating the robustness of the system using Monte Carlo-based methods. These approaches require a large number of samples, which results in high computational cost and long waiting times. To address the computational inefficiency of existing approaches, we employ Polynomial Chaos Expansion (PCE) as a rigorous method for uncertainty propagation and further extend its use to robust performance estimation. The aim is to assess if the software system is robust, i.e., it can withstand possible changes in parameter values, and continue to meet performance requirements. PCE is a very efficient technique, and requires significantly less computations to accurately estimate the distribution of performance indices. Through three very different case studies from different phases of software development and heterogeneous application domains, we show that PCE can accurately (>97%) estimate the robustness of various performance indices, and saves up to 225 hours of performance evaluation time when compared to Monte Carlo Simulation.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

03/29/2018

Robustness of the Sobol' indices to distributional uncertainty

Global sensitivity analysis (GSA) is used to quantify the influence of u...
12/17/2018

Robustness of the Sobol' indices to marginal distribution uncertainty

Global sensitivity analysis (GSA) quantifies the influence of uncertain ...
07/29/2019

Enhanced Multi-Index Monte Carlo by means of Multiple Semi-Coarsened Multigrid for Anisotropic Diffusion Problems

In many models used in engineering and science, material properties are ...
07/24/2019

Multilevel Monte Carlo Simulations of Composite Structures with Uncertain Manufacturing Defects

By adopting a Multilevel Monte Carlo (MLMC) framework, we show that only...
01/19/2020

Distributionally Robust Bayesian Quadrature Optimization

Bayesian quadrature optimization (BQO) maximizes the expectation of an e...
07/15/2020

Hardware Acceleration of Monte-Carlo Sampling for Energy Efficient Robust Robot Manipulation

Algorithms based on Monte-Carlo sampling have been widely adapted in rob...
01/18/2019

An Efficient Monte Carlo-based Probabilistic Time-Dependent Routing Calculation Targeting a Server-Side Car Navigation System

Incorporating speed probability distribution to the computation of the r...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.