Formal Verification of Cyber-Physical Systems using Theorem Proving (Invited Paper)

03/08/2020 ∙ by Adnan Rashid, et al. ∙ Concordia University NUST 0

Due to major breakthroughs in software and engineering technologies, embedded systems are increasingly being utilized in areas ranging from aerospace and next-generation transportation systems, to smart grid and smart cities, to health care systems, and broadly speaking to what is known as Cyber-Physical Systems (CPS). A CPS is primarily composed of several electronic, communication and controller modules and some actuators and sensors. The mix of heterogeneous underlying smart technologies poses a number of technical challenges to the design and more severely to the verification of such complex infrastructure. In fact, a CPS shall adhere to strict safety, reliability, performance and security requirements, where one needs to capture both physical and random aspects of the various CPS modules and then analyze their interrelationship across interlinked continuous and discrete dynamics. Often times however, system bugs remain uncaught during the analysis and in turn cause unwanted scenarios that may have serious consequences in safety-critical applications. In this paper, we introduce some of the challenges surrounding the design and verification of contemporary CPS with the advent of smart technologies. In particular, we survey recent developments in the use of theorem proving, a formal method, for the modeling, analysis and verification of CPS, and overview some real world CPS case studies from the automotive, avionics and healthtech domains from system level to physical components.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Cyber-Physical systems (CPS) [74] are engineered systems involving a cyber component that controls the physical components, as shown in Figure 1. The cyber elements include embedded systems and network controllers, which are usually modeled as discrete events. Whereas, the physical components exhibit continuous dynamics, such as the physical motion of a robot in space or the working of an analog circuit, and are commonly modeled using differential equations. CPS are capable of performing two main functionalities (a) constructing the cyber space using intelligent data management, computational and analytical capabilities; and (b) real-time data acquisition from the physical world and information feedback from the cyber space using some advanced connectivity, as depicted in Figure 1. They can be small, such as artificial pancreas, or very large and complex, such as a smart car or smart energy grid. The development of powerful embedded system hardware, low-power sensing and widely deployed communication networks has drastically increased the dependence of system functionality on CPS. CPS are widely used in advanced automotive systems (autonomous vehicles and smart cars), avionics, medical systems and devices, optical systems, industrial process control, smart grids, traffic safety and control, robotics and telecommunication networks, etc. For example, smart (self-driving) cars are considered as a highly complex autonomous CPS composed of over one hundred processors, and an array of sensors and actuators that interact with the external environment, like the road infrastructure and internet.

Figure 1: Components of a CPS [2]

The main goals for an efficient design of CPS are to co-design its cyber and physical parts, and to engineer the system of systems involving the intrinsic heterogeneity. Moreover, an increase in the complexity of its various components and the utilization of advanced technologies pose a major challenge for developing a CPS. For example, in the case of smart cars, it is required to develop cost-effective methods ensuring: a) design and analysis (verification) of its various components at different levels of abstraction, i.e., at different systems and software architecture levels; b) analyzing and understanding the interactions of system of systems, e.g., cars’ control system and its various components, such as engine, wheel, steering; c) minimizing the cost of the car by ensuring the safety, reliability, performance and stability of the overall system. Thus, these requirements have to be fulfilled for the efficient design and analysis of a CPS.

The analysis of CPS can generally be characterised as of three types, namely, functional, performance and dependability analysis. For example, the functional analysis involves the analysis of the physical, control and signal processing components of CPS. Each of these characteristics also need to consider a hybrid behavior incorporating both continuous and discrete dynamics, e.g., the physical and cyber elements of the underlying system.

Conventionally, CPS are analyzed using paper-and-pencil methods or computer-based numerical and symbolic techniques. Moreover, most of the time is spent on designing the life-cycle of CPS and their physical (dynamical) behaviour needs to be manipulated. However, there is a lack of theoretical foundations for CPS dynamics and compositional theories for the heterogeneous systems in the tools associated with these analyses. Moreover, these analysis methods suffer from their inherent limitations, like human-error proneness, discretization and numerical errors and the usage of unverified simplification algorithms [23] and thus cannot provide absolute accuracy of the corresponding analysis. Due to the safety critical-nature of CPS, the accuracy of their design and analysis is becoming a dire need. For example, the fatal crash of Uber’s self-driving car in March that killed a pedestrian in Tempe, Arizona, USA was found to be caused by some sensor’s anomalies [1]. A more rigourous analysis of CPS could have avoided this incident.

Formal methods [45] have been used as a complementary technique for analyzing CPS and thus can overcome the above-mentioned inaccuracy limitations of the analysis. The two most commonly used formal methods are model checking [14] and theorem proving [35]. Model checking is based on developing a state-space based model of the underlying system and formally verifying the properties of interest, specified in temporal logic. It has been used for analyzing several aspects of a CPS [21]. However, this kind of analysis involves the discretization of the continuous dynamical models and thus compromises the accuracy of the corresponding analysis. Moreover, it also suffers from the state-space explosion problem [14]. Theorem proving [35] is a computer based mathematical method that involves developing a mathematical model of the given system in an appropriate logic and the formal verification of the properties of interest based on mathematical reasoning within the sound core of a theorem prover. The involvement of the formal model and its associated formally specified properties along with the sound nature of theorem proving ensures the accuracy and completeness of the analysis. Based on the decidability or undecidability of the underlying logic, e.g., propositional or higher-order logic, theorem proving can be automatic or interactive, respectively.

Many theorem provers, e.g., HOL4 [92], HOL Light [36], Isabelle [69], KeYmaera [73], Coq [19], PVS [68] have been used for the formal analysis (formal verification) of CPS, e,g., formal functional analysis, formal probabilistic and performance analysis, formal dependability analysis, and hybrid analysis. For instance, the KeYmaera theorem prover has been specifically designed for the formal verification of hybrid systems, thus, incorporating both the continuous and discrete dynamics of the underlying system. KeYmaera is based on deductive reasoning and computer algebraic prover technologies. It uses differential dynamic logic for the model implementation and specification of the underlying system, which is a first-order logic. Similarly, HOL Light provides an extensive support of mathematical libraries that have been used for the functional analysis, i.e., the verification of various continuous aspects of CPS, such as control systems, power electronics, electromagnetic, quantum and optical systems. HOL4 and Isabelle theorem provers provide an extensive support for the formal probabilistic and dependability analysis of systems. Likewise, Isabelle and HOL4 have been extensively used for the verification of software components, providing safety and security analysis of the underlying CPS. In this paper, we report these developments that have been done for the modeling, analysis and verification of CPS in these theorem provers.

2 Formal Functional Analysis

2.1 Verification of Physical Components

Hasan et al. [38]

proposed a framework for analyzing the optical waveguides using HOL4. In particular, the authors formally analyzed the eigenvalues for the planar optical waveguides and utilized their proposed framework for analyzing a planar asymmetric waveguide. Afshar et al. 

[5]

developed a formal support for the complex vector analysis using HOL Light and used it to formally verify the law of reflection for the planar waves. Later, the authors used the formalization of complex vectors to formalize the notions of electromagnetic optics 

[51], which is further used for performing the formal analysis of the resonant cavity enhanced photonic devices.

Siddique et al. [86] provided a formalization of geometrical optics using HOL Light. The authors formalized fundamental concepts about geometrical optics, i.e., ray, free space, optical system and its stability. Finally, they used their proposed formalization to perform the stability analysis of the Fabry-Perot resonator with fiber rod lens [82]. Next, the authors extended their framework by formalizing the ray optics of the cardinal points and utilized it for formally analyzing a thick lens [87] and the optical instrument used to compensate the ametropia of an eye [89]. Moroever, the authors formalized the notion of optical resonators and used it for formally verifying the 2-D microresonator lattice optical filters [88]. Finally, the authors extended their formal support for geometrical optics in HOL Light by performing the formal analysis of the gaussian [90] and periodic [91] optical systems.

As a part of the optics formal verification project [6], Mahmoud et al. [60] provided a support for the formal analysis of the quantum systems using HOL Light. In particular, the authors formalized the infinite dimension linear spaces and used it for formally verifying a quantum beam splitter. Next, the authors used their formalization of linear algebra to formalize the optical quantum circuits, i.e., the flip gate and used it to formally verify the beam splitter and the phase conjugating mirror [61]. Later, the authors also formalized the notion of coherent light, which is a light produced by the laser sources and formally verified its various properties using HOL Light [62]. Based on these findings, Beillahi et al. [15] proposed a framework for the hierarchical verification of the quantum circuit and used it for the formal analysis of a controlled-phase gate and the Shor’s factoring quantum circuits. Rand et al. [75] proposed a framework implementing the QWIRE quantum circuit language in Coq, which accepts a high-level abstract model of the quantum circuits and allows the verification of their properties using Coq’s features such as dependently-typed circuits and proof-carrying code. Liu et al. [54] formalized the theory of Quantum Hoare Logic (QHL) and used it for formally verifying the correctness of a nontrivial quantum algorithm using Isabelle.

2.2 Verification of Software Components

The High-Assurance Cyber Military Systems (HACMS) research program [33] was started by the Defense Advanced Research Projects Agency (DARPA) in the USA with an aim of creating a technology for constructing CPS that are resilient against cyber-attacks, i.e., CPS providing an appropriate security and safety properties. One of the major goals of this program is to create a high-assurance software for vehicles, ranging from automobiles to military vehicles, such as quadcopters and helicopters. As a part of this project, Cofer et al. [22] proposed a formal approach for constructing a secure airvehicle software to ensure security against cyber attacks using Isabelle. Moreover, the authors applied their proposed approach for formally analyzing the SMACCMcopter, which is a modified commercial quadcopter, and Boeing’s Unmanned Little Bird (ULB), which is a full-sized optionally-piloted helicopter. Klein et al. [52] presented the formal verification of seL4 microkernel in HOL4, which is a third-generation microkernel of L4 provenance. The authors formally proved that the implementation of the underlying system follows the high-level specification of the kernel behaviour using Isabelle. Moreover, they also verified two vital properties of the microkernel, i.e., 1) the kernel will not perform an unsafe operation; 2) it will never crash.

2.3 Verification of Control and Signal Processing Components

Transform methods, such as Laplace, Fourier and

-transforms are widely used for solving dynamical models and performing the frequency domain analysis of systems. Generally, the dynamics of a system in frequency domain are characterized by the transfer function and frequency response, providing a relationship between its input and output and are important properties of the control and signal processing components of a CPS. In this regards, Taqdees et al. 

[93] formalized the Laplace transform using multivariate calculus theories of HOL Light. Moreover, the authors used their formalization of the Laplace transform for formally verifying the transfer function of the Linear Transfer Converter (LTC) circuit. Next, the authors extended their framework and provided a support to formally reason about the linear analog circuits, such as Sallen-Key low-pass filters [94] by formalizing the system governing laws such as Kirchhoff’s Current Law (KCL) and Kirchhoff’s Voltage Law (KVL) using HOL Light. Later, Rashid et al. [81] proposed a new formalization of the Laplace transform based on the notion of sets and used it for analyzing the control system of the Unmanned Free-swimming Submersible (UFSS) vehicle [79] and 4- soft error crosstalk model [76]. The Laplace transform [49, 96] has also been formalized in Isabelle and Coq theorem provers. Similarly, Rashid et al. [77]

formalized the Fourier transform in HOL Light and used it to formally analyze an Automobile Suspension System (ASS), an audio equalizer, a drug therapy model and a MEMs accelerometer 

[78].

To perform the transfer function based analysis of the discrete-time systems, Siddique et al. [84] formalized -transform using HOL Light and used it for the formal analysis of Infinite Impulse Response (IIR) Digital Signal Processing (DSP) filter. Later, the authors extended their proposed framework by providing the formal support for the inverse -transform and used it for formally analyzing a switched-capacitor interleaved DC-DC voltage doubler [85]. Beillahi et al. [17] proposed a formalization of signal-flow graph, which is widely used for evaluating the system performance in the form of transfer function, using HOL Light. The authors used their proposed framework for formally analyzing a die design process [16], -boost cell interleaved DC-DC, Pulse Width Modulation (PWM) push-pull DC-DC converters [17], Double-coupler Double-ring (DCDR) photonic processor [83], z-source impedance network and PANDA Vernier resonator [18].

Farooq et al. [32] proposed a formal framework for the kinematic analysis of a two-link planar manipulator, which describes a geometrical relationship between the robotic joints and links, and is widely used to capture the motion of the robots. Moreover, the authors performed the formal kinematic analysis of a biped walking robot using HOL Light. Next, Affeldt et al. [4] carried forward this idea and formalized the foundational support for 3D analysis of the robotic manipulators in Coq. The authors used their proposed framework for the kinematic analysis of the SCARA robot manipulator. Wu et al. [97] used HOL4 to formally reason about the forward kinematics of the 3-DOF planar robot manipulator. Similarly, Li et al. [53] provided the formal verification of the Collision-free Motion Planning Algorithm (CFMPA) of Dual-arm Robot (DAR) using HOL4. Walter et al. [95] formally verified a collision-avoidance algorithm for service robots in Isabelle. The authors mainly formalized the safety zone of the robot based on the algorithm and used it to formally verify that the robot will stop upon facing an obstacle, otherwise, it will continue its movement within the safety zone. Recently, Rashid et al. [80] provided the formal modeling and analysis of the -DOF robotic cell injection systems using HOL Light.

2.4 Formal Hybrid Analysis

Platzer et al. [70] developed an algorithm for the verification of the safety properties of CPS. The authors used the notion of continuous generalization of induction to compute the differential invariants, which do not require solving the differential equations capturing the dynamics of CPS. Moreover, they used their proposed algorithm for formally verifying the collision avoidance properties in car controls and aircraft roundabout maneuvers [71] using KeYmaera. Similarly, Platzer et al. [72] verified the safety, controllability, liveness, and reactivity properties of the European Train Control System (ETCS) protocol using KeYmaera. KeYmaera has also been widely used for the dynamical analysis of various CPS, such as a distributed car control system [59], freeway traffic control [67], autonomous robotic vehicles [66] and industrial airborne collision avoidance system [50]. Recently, Bohrer et al. [20] presented VeriPhy, a verified pipeline for automatically transforming verified models of CPS to verified controller executables. It proves CPS safety at runtime by verified monitors. All these analysis performed using KeYmaera are based on the differential dynamics logic, which captures both the continuous and discrete dynamics of CPS and their interaction. This logic allows the suitable automation of the verification process as well. Similarly, Foster et al. [34] proposed a framework for the verification of CPS based on Unifying Theories of Programming (UTP) and Isabelle/HOL. In particular, the authors provide the implementation of designs, reactive processes, and the hybrid relational calculus, which are important foundational theories for analyzing CPS.

3 Formal Probabilistic and Performance Analysis

Hasan et al. [46]

proposed a higher-order logic framework for the probabilistic analysis of the systems using HOL4. The authors first formalized the standard uniform random variable 

[40]. Next, they used this random variable alongside a non-uniform random number generation method to formalize continuous uniform random variables. Finally, the authors used their proposed formalization for the probabilistic analysis of roundoff error in a digital processor [40]. Next, Hasan et al. [42]

used HOL4 for the formal verification of the expectation and variance of the discrete random variable and used their expectation theory to formally reason about the Coupon Collector’s problem 

[42]

. Later, the authors extended their framework by providing the formal verification of the expectation properties of the continuous random variables, i.e., Uniform, Triangular and Exponential 

[37]

. Next, the authors formalized the indicator random variables using HOL4 and used it for the expected time complexity analysis of various algorithms, i.e., the birthday paradox, the hat-check and the hiring problems 

[43]. Elleuch et al. [30]

used the probability theory of HOL4 to formally reason about the detection properties of Wireless Sensor Networks (WSNs) and a WSN-based monitoring framework 

[31]. Moreover, the authors conducted the performance analysis of WSNs [29]. Hasan et al. also used their probability theory in HOL4 for conducting the performance analysis of Automatic-repeat-request (ARQ) protocols, i.e., Stop-and-Wait, Go-Back-N and Selective-Repeat protocols [41]. Finally, Hasan et al. [44]

formalized the notion of conditional probability and formally verified its classical properties, i.e., Bayes’ theorem and total probability law. The authors utilized their formalization for formally analyzing the binary asymmetric channel, which is widely used in communication systems. Mhamdi et al. 

[63]

formalized the Lebesgue integral using HOL4 and used it for formally verifying the Markov and Chebyshev inequalities, and the Weak Law of Large Numbers (WLLN) theorem. Next, the authors built upon Lebesgue integral to formalize the Radon-Nikodym derivative and used it for formalizing the fundamentals of information theory, i.e., Shannon and relative entropies 

[64]. Later, Mhamdi et al. [65] used the probabilistic analysis support developed in HOL4 to evaluate the security properties of the confidentiality protocols. A library for the formal probabilistic analysis has also been developed in Isabelle. Holzl et al. [47] formalized measure theory with extended real numbers as measure values, in particular, the authors formalized Lebesgue integral, product measures and Fubini’s theorem using Isabelle. Eberl at al. [24] developed an inductive compiler, which takes programs in a probabilistic functional language and computes density functions for the probability spaces using Isabelle. Similarly, Holzl et al. [48]

proposed a formalization of Markov chains and used it to formally verify the ZeroConf and the Crowds protocols using Isabelle.

4 Formal Dependability Analysis

Hasan et al. [39] formalized some fundamental concepts about the reliability theory in HOL4 and used it for formal reliability analysis of reconfigurable memory arrays in the presence of stuck-at and coupling faults. Moreover, the authors performed the reliability analysis of the combinational circuits, such as full adders, comparators and multiplier. Later, Abbasi et al. [3]

extended the reliability analysis framework by formally verifying some statistical properties, i.e., second moment and variance and other reliability concepts, i.e., survival, hazard and fractile functions. The authors utilized their proposed framework for formally analyzing the essential electronic and electrical system components.

Liu et al. [56] proposed a framework to reason about the finite-state discrete-time Markov chains using HOL4 and formally verified some of its properties such as joint and steady-state probabilities, and reversibility. The authors utilized their proposed framework to formally analyze a binary communication channel and an automatic mail quality measurement protocol [58]. Next, the authors formalized the discrete-time Markov reward models and used it to formally reason about the memory contention problem of a multi-processor system [57]

. Later, the authors proposed a framework to formally reason about the properties of the Hidden Markov Models (HMMs) such as joint probabilities and formally analyzed a DNA sequence 

[55].

Ahmad et al. [9] developed a higher-order logic based framework for the formal dependability analysis using probability theory of HOL4. The proposed analysis provides the failure characteristics of the systems, i.e., reliability, availability, maintainability, etc. The authors formalized the Reliability Block Diagrams (RBD) [7], which are the graphical representations providing the functional behaviour of a system modules and their interconnections. The proposed formalization of RBD has been used for formally analyzing a simple oil and gas pipeline, a generic Virtual Data Center (VDC) [13], Reliable Multi-Segment Transport (RMST) data transport, Event to Sink Reliable Transport (ESRT) protocols [12] and Logistics Service Supply Chains (LSSCs) [8]. Similarly, Ahmad et al. [10] proposed a framework for the formal fault tree analysis using HOL4. The authors formalized the fault tree gates, i.e., AND, OR, NAND, NOR, XOR and NOT and formally verified their generic expressions for probabilities failures. Moreover, their proposed framework was used to perform the fault tree analysis of a solar array, which is used as a major source of power in the Dong Fang Hong-3 (DFH-3) satellite [10] and a communication gateway software for the next generation Air Traffic Management System (ATMS) [11].

Elderhalli et al. [26] developed a higher-order logic based framework for the formal dynamic dependability analysis using HOL4. The proposed analysis provides the dynamic failure characteristics of the systems, i.e., dynamic reliability and fault trees, etc. The authors formalized the Dynamic Fault Trees (DFTs) [25] and Dynamic Reliability Block Diagrams (DRBD) [27] using HOL4. Moreover, they used their proposed formalization for formally analyzing the Drive-by-wire System (DBW), a Shuffle-exchange Network (SEN) and Cardiac Assist System (CAS) [28].

5 Theorem Proving Support for CPS

Table 1 summarizes the formal libraries that are available in various theorem provers for performing the formal analysis of CPS. For example, the formal support for the dependability analysis of systems is only available in HOL4. Similarly, the libraries to formally reason about robotics and software components are available in most of the theorem provers. KeyMaera provides a support for formally analyzing the hybrid systems. Moreover, HOL4 and Isabelle theorem provers have a quite dense library for probabilistic and performance analyses of systems. Similarly, the transform methods are partially available in Isabelle, Coq and HOL4 theorem provers, i.e., only the Laplace transform is formalized in these theorem provers. However, HOL Light contains formal libraries for most of the transform methods, i.e., Laplace, Fourier and -transforms. Also, the formal library for analyzing the optical systems is only available in HOL Light.

Analysis/Theorem Provers HOL4 HOL light Isabelle/HOL Coq PVS Keymaera
Transform Methods
Probabilistic Analysis
Performance Analysis
Dependability Analysis
Hybrid Systems
Optical Systems
Quantum Systems
Robotic Systems
Software Components
Table 1: Libraries for Formal Analysis in Major Theorem Provers

6 Conclusion

CPS are highly complex systems composed of actuators, sensors, and several electronic, communication and controller modules, and exhibit both the continuous and discrete dynamics. Due to the safety critical-nature of CPS, their accurate analysis is of utmost importance. This paper surveys some of the efforts that have been done regarding the formal verification of CPS using theorem proving by highlighting the aspects of CPS that have been verified using different theorem provers. In this regard, only one dedicated theorem prover, KeYmaera, has been developed for analyzing hybrid systems. However, we need to develop dedicated formal libraries in other theorem provers that can support the analysis of hybrid systems, i.e., incorporating the interlinked discrete and continuous-time features of a CPS simultaneously.

References

  • [1] (2018) . Note: https://arstechnica.com/tech-policy/2018/05/report-software-bug-led-to-death-in-ubers-self-driving-crash/?amp=1 Cited by: §1.
  • [2] (2020) . Note: https://www.2b1stconsulting.com/cyber-physical-systems-cps/ Cited by: Figure 1.
  • [3] N. Abbasi, O. Hasan, and S. Tahar (2014) An Approach for Lifetime Reliability Analysis using Theorem Proving. Journal of Computer and System Sciences 80 (2), pp. 323–345. Cited by: §4.
  • [4] R. Affeldt and C. Cohen (2017) Formal Foundations of 3D Geometry to Model Robot Manipulators. In Certified Programs and Proofs, pp. 30–42. Cited by: §2.3.
  • [5] S. K. Afshar, V. Aravantinos, O. Hasan, and S. Tahar (2014) Formalization of Complex Vectors in Higher-order Logic. In Intelligent Computer Mathematics, LNCS, Vol. 8543, pp. 123–137. Cited by: §2.1.
  • [6] S. K. Afshar, U. Siddique, M. Y. Mahmoud, V. Aravantinos, O. Seddiki, O. Hasan, and S. Tahar (2014) Formal Analysis of Optical Systems. Mathematics in Computer Science 8 (1), pp. 39–70. Cited by: §2.1.
  • [7] S. Ahmad, O. Hasan, and U. Siddique (2014) Towards Formal Reasoning about Molecular Pathways in HOL. In WETICE Conference (WETICE), 2014 IEEE 23rd International, pp. 378–383. Cited by: §4.
  • [8] W. Ahmad, O. Hasan, S. Tahar, and M. Hamdi (2015) Towards Formal Reliability Analysis of Logistics Service Supply Chains using Theorem Proving. In Implementation of Logics, pp. 111––121. Cited by: §4.
  • [9] W. Ahmad, O. Hasan, and S. Tahar (2016) Formal Dependability Modeling and Analysis: A Survey. In Intelligent Computer Mathematics, LNCS, Vol. 9791, pp. 132–147. Cited by: §4.
  • [10] W. Ahmad and O. Hasan (2015) Towards Formal Fault Tree Analysis using Theorem Proving. In Intelligent Computer Mathematics, LNCS, Vol. 9150, pp. 39–54. Cited by: §4.
  • [11] W. Ahmad and O. Hasan (2016) Formalization of Fault Trees in Higher-order Logic: A Deep Embedding Approach. In Dependable Software Engineering: Theories, Tools, and Applications, LNCS, Vol. 9984, pp. 264–279. Cited by: §4.
  • [12] W. Ahmed, O. Hasan, and S. Tahar (2015) Formal Reliability Analysis of Wireless Sensor Network Data Transport Protocols using HOL. In Wireless and Mobile Computing, Networking and Communications, pp. 217–224. Cited by: §4.
  • [13] W. Ahmed, O. Hasan, and S. Tahar (2016) Formalization of Reliability Block Diagrams in Higher-order Logic. Journal of Applied Logic 18, pp. 19–41. Cited by: §4.
  • [14] C. Baier, J. P. Katoen, and K. G. Larsen (2008) Principles of Model Checking. MIT press. Cited by: §1.
  • [15] S. M. Beillahi, M. Y. Mahmoud, and S. Tahar (2016) Hierarchical verification of quantum circuits. In NASA Formal Methods Symposium, LNCS, Vol. 9690, pp. 344–352. Cited by: §2.1.
  • [16] S. M. Beillahi, U. Siddique, and S. Tahar (2014) Towards the Application of Formal Methods in Process Engineering. Fun With Formal Methods, pp. 1–11. Cited by: §2.3.
  • [17] S. M. Beillahi, U. Siddique, and S. Tahar (2015) Formal Analysis of Power Electronic Systems. In Formal Methods and Software Engineering, LNCS, Vol. 9407, pp. 270–286. Cited by: §2.3.
  • [18] S. M. Beillahi, U. Siddique, and S. Tahar (2016) Formal Analysis of Engineering Systems Based on Signal-Flow-Graph Theory. In Numerical Software Verification, LNCS, Vol. 10152, pp. 31–46. Cited by: §2.3.
  • [19] Y. Bertot and P. Castéran (2013) Interactive Theorem Proving and Program Development: Coq’Art: the Calculus of Inductive Constructions. Springer Science & Business Media. Cited by: §1.
  • [20] B. Bohrer, Y. K. Tan, S. Mitsch, M. O. Myreen, and A. Platzer (2018) VeriPhy: Verified Controller Executables from Verified Cyber-physical System Models. In Programming Language Design and Implementation, pp. 617–630. Cited by: §2.4.
  • [21] E. M. Clarke and P. Zuliani (2011) Statistical Model Checking for Cyber-physical Systems. In Automated Technology for Verification and Analysis, LNCS, Vol. 6996, pp. 1–12. Cited by: §1.
  • [22] D. Cofer, A. Gacek, J. Backes, M. W. Whalen, L. Pike, A. Foltzer, M. Podhradsky, G. Klein, I. Kuz, J. Andronick, et al. (2018) A Formal Approach to Constructing Secure Air Vehicle Software. Computer 51 (11), pp. 14–23. Cited by: §2.2.
  • [23] A. J. Durán, M. Pérez, and J. L. Varona (2013) Misfortunes of a Mathematicians’ Trio using Computer Algebra Systems: Can We Trust?. CoRR abs/1312.3270. Cited by: §1.
  • [24] M. Eberl, J. Hölzl, and T. Nipkow (2015)

    A Verified Compiler for Probability Density Functions

    .
    In European Symposium on Programming Languages and Systems, pp. 80–104. Cited by: §3.
  • [25] Y. Elderhalli, W. Ahmad, O. Hasan, and S. Tahar (2019) Probabilistic Analysis of Dynamic Fault Trees using HOL Theorem Proving. Journal of Applied Logics—IfCoLog Journal of Logics and their Applications 6 (3). Cited by: §4.
  • [26] Y. Elderhalli, O. Hasan, W. Ahmad, and S. Tahar (2018) Formal Dynamic Fault Trees Analysis using an Integration of Theorem Proving and Model Checking. In NASA Formal Methods Symposium, LNCS, Vol. 10811, pp. 139–156. Cited by: §4.
  • [27] Y. Elderhalli, O. Hasan, and S. Tahar (2019) A Formally Verified Algebraic Approach for Dynamic Reliability Block Diagrams. In Formal Methods and Software Engineering, pp. 253–269. Cited by: §4.
  • [28] Y. Elderhalli, O. Hasan, and S. Tahar (2019) A Methodology for the Formal Verification of Dynamic Fault Trees Using HOL Theorem Proving. IEEE Access 7, pp. 136176–136192. Cited by: §4.
  • [29] M. Elleuch, O. Hasan, S. Tahar, and M. Abid (2013) Towards the Formal Performance Analysis of Wireless Sensor Networks. In Enabling Technologies: Infrastructure for Collaborative Enterprises, pp. 365–370. Cited by: §3.
  • [30] M. Elleuch, O. Hasan, S. Tahar, and M. Abid (2015) Formal Probabilistic Analysis of Detection Properties in Wireless Sensor Networks. Formal Aspects of Computing 27 (1), pp. 79–102. Cited by: §3.
  • [31] M. Elleuch, O. Hasan, S. Tahar, and M. Abid (2016) Formal Probabilistic Analysis of a WSN-Based Monitoring Framework for IoT Applications. In Formal Techniques for Safety-Critical Systems, CCIS, Vol. 694, pp. 93–108. Cited by: §3.
  • [32] B. Farooq, O. Hasan, and S. Iqbal (2013) Formal Kinematic Analysis of the Two-link Planar Manipulator. In Formal Methods and Software Engineering, LNCS, Vol. 8144, pp. 347–362. Cited by: §2.3.
  • [33] K. Fisher, J. Launchbury, and R. Richards (2017) The HACMS Program: using Formal Methods to Eliminate Exploitable Bugs. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 375 (2104), pp. 20150401. Cited by: §2.2.
  • [34] S. Foster and J. Woodcock (2017) Towards Verification of Cyber-Physical Systems with UTP and Isabelle/HOL. In Concurrency, Security, and Puzzles, LNCS, Vol. 10160, pp. 39–64. Cited by: §2.4.
  • [35] J. Harrison (2009)

    Handbook of Practical Logic and Automated Reasoning

    .
    Cambridge University Press. Cited by: §1.
  • [36] J. Harrison (1996) HOL Light: A Tutorial Introduction. In Formal Methods in Computer-Aided Design, LNCS, Vol. 1166, pp. 265–269. Cited by: §1.
  • [37] O. Hasan, N. Abbasi, B. Akbarpour, S. Tahar, and R. Akbarpour (2009) Formal Reasoning about Expectation Properties for Continuous Random Variables. In Formal Methods, LNCS, Vol. 5850, pp. 435–450. Cited by: §3.
  • [38] O. Hasan, S. K. Afshar, and S. Tahar (2009) Formal Analysis of Optical Waveguides in HOL. In Theorem Proving in Higher Order Logics, LNCS, Vol. 5674, pp. 228–243. Cited by: §2.1.
  • [39] O. Hasan, S. Tahar, and N. Abbasi (2010) Formal Reliability Analysis using Theorem Proving. IEEE Transactions on Computers 59 (5), pp. 579–592. Cited by: §4.
  • [40] O. Hasan and S. Tahar (2007) Formalization of the Standard Uniform Random Variable. Theoretical Computer Science 382 (1), pp. 71–83. Cited by: §3.
  • [41] O. Hasan and S. Tahar (2008) Performance Analysis of ARQ Protocols using a Theorem Prover. In Performance Analysis of Systems and Software, pp. 85–94. Cited by: §3.
  • [42] O. Hasan and S. Tahar (2008) Using Theorem Proving to Verify Expectation and Variance for Discrete Random Variables. Journal of Automated Reasoning 41 (3-4), pp. 295–323. Cited by: §3.
  • [43] O. Hasan and S. Tahar (2010) Formally Analyzing Expected Time Complexity of Algorithms Using Theorem Proving. Journal of Computer Science and Technology 25 (6), pp. 1305–1320. Cited by: §3.
  • [44] O. Hasan and S. Tahar (2011) Reasoning about Conditional Probabilities in a Higher-order-logic Theorem Prover. Journal of Applied Logic 9 (1), pp. 23–40. Cited by: §3.
  • [45] O. Hasan and S. Tahar (2015) Formal Verification Methods. Encyclopedia of Information Science and Technology, IGI Global Publication, pp. 7162–7170. Cited by: §1.
  • [46] O. Hasan and S. Tahar (2015) Formalized Probability Theory and Applications using Theorem Proving. IGI Global. Cited by: §3.
  • [47] J. Hölzl and A. Heller (2011) Three Chapters of Measure Theory in Isabelle/HOL. In Interactive Theorem Proving, pp. 135–151. Cited by: §3.
  • [48] J. Hölzl and T. Nipkow (2012) Interactive Verification of Markov Chains: Two Distributed Protocol Case Studies. arXiv preprint arXiv:1212.3870. Cited by: §3.
  • [49] F. Immler (2018) Laplace Transform - Archive of Formal Proofs. Note: https://www.isa-afp.org/entries/Laplace_Transform.html Cited by: §2.3.
  • [50] J. B. Jeannin, K. Ghorbal, Y. Kouskoulas, R. Gardner, A. Schmidt, E. Zawadzki, and A. Platzer (2015) Formal Verification of ACAS X, An Industrial Airborne Collision Avoidance System. In Embedded Software, pp. 127–136. Cited by: §2.4.
  • [51] S. Khan-Afshar, O. Hasan, and S. Tahar (2014) Formal Analysis of Electromagnetic Optics. In Novel Optical Systems Design and Optimization XVII, Vol. 9193, pp. 91930A. Cited by: §2.1.
  • [52] G. Klein, K. Elphinstone, G. Heiser, J. Andronick, D. Cock, P. Derrin, D. Elkaduwe, K. Engelhardt, R. Kolanski, M. Norrish, et al. (2009) SeL4: Formal Verification of an OS Kernel. In Operating Systems Principles, pp. 207–220. Cited by: §2.2.
  • [53] L. Li, Z. Shi, Y. Guan, C. Zhao, J. Zhang, and H. Wei (2014) Formal Verification of a Collision-free Algorithm of Dual-arm Robot in HOL4. In Robotics and Automation, pp. 1380–1385. Cited by: §2.3.
  • [54] J. Liu, B. Zhan, S. Wang, S. Ying, T. Liu, Y. Li, M. Ying, and N. Zhan (2019) Formal Verification of Quantum Algorithms using Quantum Hoare Logic. In Computer Aided Verification, pp. 187–207. Cited by: §2.1.
  • [55] L. Liu, V. Aravantinos, O. Hasan, and S. Tahar (2014) On the Formal Analysis of HMM using Theorem Proving. In Formal Methods and Software Engineering, LNCS, Vol. 8829, pp. 316–331. Cited by: §4.
  • [56] L. Liu, O. Hasan, and S. Tahar (2011) Formalization of Finite-state Discrete-time Markov Chains in HOL. In Automated Technology for Verification and Analysis, LNCS, Vol. 6996, pp. 90–104. Cited by: §4.
  • [57] L. Liu, O. Hasan, and S. Tahar (2013) Formal Analysis of Memory Contention in a Multiprocessor System. In Formal Methods: Foundations and Applications, LNCS, Vol. 8195, pp. 195–210. Cited by: §4.
  • [58] L. Liu, O. Hasan, and S. Tahar (2013) Formal Reasoning about Finite-state Discrete-time Markov Chains in HOL. Journal of Computer Science and Technology 28 (2), pp. 217–231. Cited by: §4.
  • [59] S. M. Loos, A. Platzer, and L. Nistor (20011) Adaptive Cruise Control: Hybrid, Distributed, and now Formally Verified. In Formal Methods, LNCS, Vol. 6664, pp. 42–56. Cited by: §2.4.
  • [60] M. Y. Mahmoud, V. Aravantinos, and S. Tahar (2013) Formalization of Infinite Dimension Linear Spaces with Application to Quantum Theory. In NASA Formal Methods Symposium, LNCS, Vol. 7871, pp. 413–427. Cited by: §2.1.
  • [61] M. Y. Mahmoud, V. Aravantinos, and S. Tahar (2014) Formal verification of Optical Quantum Flip Gate. In Interactive Theorem Proving, LNCS, Vol. 8558, pp. 358–373. Cited by: §2.1.
  • [62] M. Y. Mahmoud and S. Tahar (2014) On the Quantum Formalization of Coherent Light in HOL. In NASA Formal Methods Symposium, LNCS, Vol. 8430, pp. 128–142. Cited by: §2.1.
  • [63] T. Mhamdi, O. Hasan, and S. Tahar (2010) On the Formalization of the Lebesgue Integration Theory in HOL. In Interactive Theorem Proving, LNCS, Vol. 6172, pp. 387–402. Cited by: §3.
  • [64] T. Mhamdi, O. Hasan, and S. Tahar (2011) Formalization of Entropy Measures in HOL. In Interactive Theorem Proving, LNCS, Vol. 6898, pp. 233–248. Cited by: §3.
  • [65] T. Mhamdi, O. Hasan, and S. Tahar (2015) Evaluation of Anonymity and Confidentiality Protocols using Theorem Proving. Formal Methods in System Design 47 (3), pp. 265–286. Cited by: §3.
  • [66] S. Mitsch, K. Ghorbal, and A. Platzer (2013) On Provably Safe Obstacle Avoidance for Autonomous Robotic Ground Vehicles. In Robotics: Science and Systems, Cited by: §2.4.
  • [67] S. Mitsch, S. M. Loos, and A. Platzer (2012) Towards Formal Verification of Freeway Traffic Control. In Cyber-Physical Systems, pp. 171–180. Cited by: §2.4.
  • [68] S. Owre, J. M. Rushby, and N. Shankar (1992) PVS: A Prototype Verification System. In Automated Deduction, LNCS, Vol. 607, pp. 748–752. Cited by: §1.
  • [69] L. C. Paulson (1994) Isabelle: A Generic Theorem Prover. Vol. 828, Springer Science & Business Media. Cited by: §1.
  • [70] A. Platzer and E. M. Clarke (2009) Computing Differential Invariants of Hybrid Systems as Fixedpoints. Formal Methods in System Design 35 (1), pp. 98–120. Cited by: §2.4.
  • [71] A. Platzer and E. M. Clarke (2009) Formal Verification of Curved Flight Collision Avoidance Maneuvers: A Case Study. In Formal Methods, LNCS, Vol. 5850, pp. 547–562. Cited by: §2.4.
  • [72] A. Platzer and J. D. Quesel (2009) European Train Control System: A Case Study in Formal Verification. In Formal Methods and Software Engineering, LNCS, Vol. 5885, pp. 246–265. Cited by: §2.4.
  • [73] A. Platzer and J. Quesel (2008) KeYmaera: A Hybrid Theorem Prover for Hybrid Systems (System Description). In Automated Reasoning, LNCS, Vol. 5195, pp. 171–178. Cited by: §1.
  • [74] R. Rajkumar, I. Lee, L. Sha, and J. Stankovic (2010) Cyber-physical Systems: The Next Computing Revolution. In Design Automation Conference, pp. 731–736. Cited by: §1.
  • [75] R. Rand, J. Paykin, and S. Zdancewic (2018) QWIRE practice: Formal Verification of Quantum Circuits in Coq. arXiv preprint arXiv:1803.00699. Cited by: §2.1.
  • [76] A. Rashid and O. Hasan (2018) Formalization of Lerch’s Theorem using HOL Light. Journal of Applied Logics—IFCoLog Journal of Logics and their Applications 5 (8), pp. 1623–1652. Cited by: §2.3.
  • [77] A. Rashid and O. Hasan (2016) On the Formalization of Fourier Transform in Higher-order Logic. In Interactive Theorem Proving, LNCS, Vol. 9807, pp. 483–490. Cited by: §2.3.
  • [78] A. Rashid and O. Hasan (2017) Formal Analysis of Continuous-time Systems using Fourier Transform. arXiv preprint arXiv:1707.09941. Cited by: §2.3.
  • [79] A. Rashid and O. Hasan (2017) Formal Analysis of Linear Control Systems using Theorem Proving. In Formal Methods and Software Engineering, LNCS, Vol. 10610, pp. 345–361. Cited by: §2.3.
  • [80] A. Rashid and O. Hasan (2017) Formal Analysis of Robotic Cell Injection Systems using Theorem Proving. In Design, Modeling, and Evaluation of Cyber Physical Systems, LNCS, Vol. 11267, pp. 127–141. Cited by: §2.3.
  • [81] A. Rashid and O. Hasan (2017) Formalization of Transform Methods using HOL Light. In Intelligent Computer Mathematics, LNAI, Vol. 10383, pp. 319–332. Cited by: §2.3.
  • [82] U. Siddique, V. Aravantinos, and S. Tahar (2013) Formal Stability Analysis of Optical Resonators. In NASA Formal Methods Symposium, LNCS, Vol. 7871, pp. 368–382. Cited by: §2.1.
  • [83] U. Siddique, S. M. Beillahi, and S. Tahar (2015) On the Formal Analysis of Photonic Signal Processing Systems. In Formal Methods for Industrial Critical Systems, LNCS, Vol. 9128, pp. 162–177. Cited by: §2.3.
  • [84] U. Siddique, M. Y. Mahmoud, and S. Tahar (2014)

    On the Formalization of Z-Transform in HOL

    .
    In Interactive Theorem Proving, LNCS, Vol. 8558, pp. 483–498. Cited by: §2.3.
  • [85] U. Siddique, M. Y. Mahmoud, and S. Tahar (2018) Formal Analysis of Discrete-time Systems using Z-transform. Journal of Applied Logics—IFCoLog Journal of Logics and their Applications 5 (4). Cited by: §2.3.
  • [86] U. Siddique and S. Tahar (2014) A Framework for Formal Reasoning about Geometrical Optics. In Intelligent Computer Mathematics, LNCS, Vol. 8543, pp. 453–456. Cited by: §2.1.
  • [87] U. Siddique and S. Tahar (2014) Towards Ray Optics Formalization of Optical Imaging Systems. In Information Reuse and Integration, pp. 378–385. Cited by: §2.1.
  • [88] U. Siddique and S. Tahar (2014) Towards the Formal Analysis of Microresonators based Photonic Systems. In Design, Automation & Test in Europe, pp. 151151. Cited by: §2.1.
  • [89] U. Siddique and S. Tahar (2015) On the Formalization of Cardinal Points of Optical Systems. In Formalisms for Reuse and Systems Integration, AISC, Vol. 346, pp. 79–102. Cited by: §2.1.
  • [90] U. Siddique and S. Tahar (2016) On the Formal Analysis of Gaussian Optical Systems in HOL. Formal Aspects of Computing 28 (5), pp. 881–907. Cited by: §2.1.
  • [91] U. Siddique and S. Tahar (2017) Formal Verification of Stability and Chaos in Periodic Optical Systems. Journal of Computer and System Sciences 88, pp. 271–289. Cited by: §2.1.
  • [92] K. Slind and M. Norrish (2008) A Brief Overview of HOL4. In Theorem Proving in Higher Order Logics, TPHOLs, Lecture Notes in Computer Science, Vol. 5170, pp. 28–32. Cited by: §1.
  • [93] S. H. Taqdees and O. Hasan (2013) Formalization of Laplace Transform Using the Multivariable Calculus Theory of HOL-Light. In

    Logic for Programming, Artificial Intelligence, and Reasoning

    ,
    LNCS, Vol. 8312, pp. 744–758. Cited by: §2.3.
  • [94] S. H. Taqdees and O. Hasan (2017) Formally Verifying Transfer Functions of Linear Analog Circuits. IEEE Design & Test 34 (5), pp. 30–37. Cited by: §2.3.
  • [95] D. Walter, H. Täubig, and C. Lüth (2010) Experiences in Applying Formal Verification in Robotics. In Computer Safety, Reliability, and Security, LNCS, Vol. 6351, pp. 347–360. Cited by: §2.3.
  • [96] Y. Wang and G. Chen (2017) Formalization of Laplace Transform in Coq. In Dependable Systems and Their Applications, pp. 13–21. Cited by: §2.3.
  • [97] A. Wu, Z. Shi, X. Yang, Y. Guan, Y. Li, and X. Song (2017) Formalization and Analysis of Jacobian Matrix in Screw Theory and its Application in Kinematic Singularity. In Intelligent Robots and Systems, pp. 2835–2842. Cited by: §2.3.