DeepAI AI Chat
Log In Sign Up

DoKnowMe: Towards a Domain Knowledge-driven Methodology for Performance Evaluation

by   Zheng Li, et al.

Software engineering considers performance evaluation to be one of the key portions of software quality assurance. Unfortunately, there seems to be a lack of standard methodologies for performance evaluation even in the scope of experimental computer science. Inspired by the concept of "instantiation" in object-oriented programming, we distinguish the generic performance evaluation logic from the distributed and ad-hoc relevant studies, and develop an abstract evaluation methodology (by analogy of "class") we name Domain Knowledge-driven Methodology (DoKnowMe). By replacing five predefined domain-specific knowledge artefacts, DoKnowMe could be instantiated into specific methodologies (by analogy of "object") to guide evaluators in performance evaluation of different software and even computing systems. We also propose a generic validation framework with four indicators (i.e. usefulness, feasibility, effectiveness and repeatability), and use it to validate DoKnowMe in the Cloud services evaluation domain. Given the positive and promising validation result, we plan to integrate more common evaluation strategies to improve DoKnowMe and further focus on the performance evaluation of Cloud autoscaler systems.


page 1

page 2

page 3

page 4


Criteria Based Evaluation Framework for Service Oriented Methodologies

Service Oriented Software Engineering is based on concepts and principle...

Proposition of an implementation framework enabling benchmarking of Holonic Manufacturing Systems

Performing an overview of the benchmarking initiatives oriented towards ...

PROMETHEUS: PROcedural METhodology for developing HEuristics of USability

Usability is used to assess the effectiveness of a software product from...

Should we Reload Time Series Classification Performance Evaluation ? (a position paper)

Since the introduction and the public availability of the ucr time serie...

Domain Knowledge Discovery Guided by Software Trace Links

Software-intensive projects are specified and modeled using domain termi...

GENOME: A GENeric methodology for Ontological Modelling of Epics

Ontological knowledge modelling of epics, though being an established re...

The State of Research on Function-as-a-Service Performance Evaluation: A Multivocal Literature Review

Function-as-a-Service (FaaS) is one form of the serverless cloud computi...