Systematic Evaluation and Usability Analysis of Formal Tools for System Design

by   Alessio Ferrari, et al.

Formal methods and supporting tools have a long record of successes in the development of safety-critical systems. However, it cannot be said that a single tool has emerged as the dominant solution for system design. Each tool differs from the others in terms of modelling language used, verification capabilities and other complementary features, and each development context has peculiar needs that require different tools. This is particularly problematic for the railway industry, in which formal methods are highly recommended by the norms, but no actual guidance is provided for the selection of tools. To guide companies in the selection of the most appropriate formal tools to adopt for their contexts, a clear assessment of the features of the available tools is required. To address this goal, this paper considers a set of 14 formal tools for system design, and presents a systematic evaluation of the tools and a usability analysis with practitioners. Results are discussed considering the most desired aspects by industry and previous related work. The focus is on the railway domain, but the overall methodology can be applied to similar contexts. Our study contributes with a systematic evaluation of formal tools and shows that despite the poor graphical interfaces, usability and maturity of the tools are not major problems, as claimed by other works. Instead, process integration support is the most relevant pain point for the majority of the platforms.


page 1

page 8

page 9


Towards making formal methods normal: meeting developers where they are

Formal verification of software is a bit of a niche activity: it is only...

Empirical Formal Methods: Guidelines for Performing Empirical Studies on Formal Methods

Empirical studies on formal methods and tools are rare. In this paper, w...

A Mathematical Framework for Evaluation of SOAR Tools with Limited Survey Data

Security operation centers (SOCs) all over the world are tasked with rea...

Tool Support of Formal Methods for Privacy by Design

Formal methods are, in principle, suited for supporting the recent parad...

A User-Study Protocol for Evaluation of Formal Verification Results and their Explanation

Context: The complexity of modern safety-critical systems in industries ...

Integrating User Design and Formal Models within PVSio-Web

Creating formal models of interactive systems has wide reaching benefits...

How UX Practitioners Produce Findings in Usability Testing

Usability testing has long been a core interest of HCI research and form...

Please sign up or login with your details

Forgot password? Click here to reset