A Methodology for Developing a Verifiable Aircraft Engine Controller from Formal Requirements

10/18/2021 ∙ by Matt Luckcuck, et al. ∙ Maynooth University 0

Verification of complex, safety-critical systems is a significant challenge. Manual testing and simulations are often used, but are only capable of exploring a subset of the system's reachable states. Formal methods are mathematically-based techniques for the specification and development of software, which can provide proofs of properties and exhaustive checks over a system's state space. In this paper, we present a formal requirements-driven methodology, applied to a model of an aircraft engine controller that has been provided by our industrial partner. Our methodology begins by formalising the controller's natural-language requirements using the (pre-existing) Formal Requirements Elicitation Tool (FRET), iteratively, in consultation with our industry partner. Once formalised, FRET can automatically translate the requirements to enable their verification alongside a Simulink model of the aircraft engine controller; the requirements can also guide formal verification using other approaches. These two parallel streams in our methodology seek to combine the results from formal requirements elicitation, classical verification approaches, and runtime verification; to support the verification of aerospace systems modelled in Simulink, from the requirements phase through to execution. Our methodology harnesses the power of formal methods in a way that complements existing verification techniques, and supports the traceability of requirements throughout the verification process. This methodology streamlines the process of developing verifiable aircraft engine controllers, by ensuring that the requirements are formalised up-front and useable during development. In this paper we give an overview of (FRET), describe our methodology and work to-date on the formalisation and verification of the requirements, and outline future work using our methodology.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The verification of complex systems presents a significant challenge to software engineering, particularly in safety-critical domains such as the aerospace sector. Industry often relies on manual testing and simulation to ensure that systems function correctly. However, these are especially time-consuming and expensive approaches that are only capable of exploring a subset of the system’s reachable states.

Formal methods are a broad range of mathematically-based approaches for software engineering and verification. Historically, industrial uptake of formal verification approaches and methods has been slow. This may be due to the complexity, usability, or scalability issues sometimes faced by formal methods. However, formal methods are widely applicable to safety-critical applications in sectors like offshore oil and gas, the nuclear industry, and space exploration [25][14][24]. They have also proven to be successful in many industrial projects [37].

Recent work argues for better integration of formal and non-formal verification  [12]. One approach is corroborative verification [36], where multiple verification techniques are applied to the system to each provide corroborative evidence that the system satisfies its requirements. This involves, for example, combining approaches that are less formal but are applied to concrete programs (such as simulation-based testing) with formal models, which often operate on more abstract representations of the system. Of course, this kind of mediation relies on a well-defined and unambiguous specification of the system’s requirements, which has long been recognised as a bottleneck in the formal verification process [34].

This paper contributes a three-phase methodology that connects a semi-formal requirements engineering phase with formal verification of a system’s design. The link between these phases also encourages traceability of the requirements through to formalised properties that can be checked against the system and its design. We demonstrate our methodology through the verification of an aircraft engine controller, the requirements and designs of which have been supplied by our industrial partner.

Our methodology starts with the system’s natural-language requirements, elicits further details, and encodes the requirements in a structured natural language. This is a an iterative process that we have performed in close collaboration with our industrial partners, who work for United Technologies Research Center in Ireland. We use the fret [15], developed by a team at the NASA Ames Research Center in the USA for this initial phase. Next, our methodology branches into two parallel phases. One phase uses requirements contracts that are automatically generated by fret to check if a Simulink diagram of the system’s design meets these requirements. The parallel phase uses the requirements expressed in fret as the starting point for formal modelling of the system, so that other formal methods may be used to verify properties of the system. In this way we enable developers to adopt alternative verification approaches guided by the formalised fret requirements. This methodology also supports a corroborative or integrated approach to verification allowing components of the system to be verified using distinct techniques. Finally, we collect the verification artefacts and results produced to assemble a verification report. The nature of this report is for the developer to decide but we include it as a phase of our methodology for completeness.

Using a structured natural language with an underlying formal semantics to encode the requirements, rather than trying to formalise them directly, enables easier communication with our industry partners while still providing a language with enough structure to reduce ambiguity. This produces a clear set of requirements that are easier to formalise than the original natural-language requirements that they are based on, while retaining the collaborative nature of the requirements elicitation process. The two parallel phases in our workflow enable the system to be verified using a combination of complimentary approaches. Overall our methodology combines requirements elicitation, formal verification of Simulink designs, and classical formal methods.

Our methodology is inspired by previous work [7], which describes using fret as the starting point for verification of Simulink designs and formal models. The contribution of this paper is to introduce a systematic methodology for driving the verification of a system’s design from formalised requirements as well as facilitating heterogeneous verification.

2 Background

Formal Methods are a set of mathematically-based techniques for verifying that a software system behaves correctly. Typical examples of formal methods include theorem provers, which provide a proof of correctness of the system’s behaviour; and model checkers, which exhaustively examine the system’s state space to check that properties are preserved in all system executions.

Dynamic properties modelling a systems safety are typically expressed in a temporal logic such as ltl or ctl [13]. These temporal logics specify changes in the state space of a system in response to events which occur next, globally, or eventually. Expressing safety properties in this way allows the use of theorem provers and model checkers to formally reason about the system’s behaviour. Properties expressed in temporal logic will be generated as part of our methodology as we connect system requirements to the formal verification of that system’s design.

Formal Methods are particularly beneficial in safety- and mission-critical domains such as aerospace, medical, transport, etc. because they provide a high degree of confidence in the correctness of the system. We aim to use these methods to verify a system’s design, through a workflow that implements our methodology, using the tools introduced in this section.

2.1 Writing Requirements in FRET

fret is an open-source tool that supports the writing of requirements in a structured natural language called

[16]. fret helps us to bridge the gap between the English-language requirements provided by our industrial partner and the formal methods that we use during verification. In turn, this link improves the traceability of the requirements throughout the verification process.

is a structured English language, where requirements take the form:

The , , and parts of the requirement are mandatory; the keyword shall states that the component’s behaviour must obey the requirement; and the other two parts of the requirement are optional. This allows requirements (described by the clause) that are restricted to a certain , are triggered by a set of s, relate to a particular of the system, may have particular constraints attached.

Figure 1 shows the ‘Create Requirement’ dialogue in fret – Figure 4 shows the dialogue in more detail, which is explained in Sect. 3. It has fields for the requirement’s ID, the ID of its parent requirement (if it has one), the project that it belongs to, and the requirement’s description in . The ‘Rationale and Comments’ item expands to allow the user to describe the requirement textually. This is useful for drafting notes and adding information that aids traceability.

Figure 1: fret’s ‘Create Requirement’ Dialogue showing an example of a requirement in fret with the requirement’s ID, the parent requirement ID, the project name, the requirement’s description in as well as text boxes recording rationale and comments.

For each requirement, fret automatically generates formalisations in both past- and future-time metric ltl. These formal translations mean that the requirements themselves can be reasoned about, formally, before the system’s development progresses any further. The fact that this translation happens automatically speeds up this cycle of checks. fret also displays a diagramatic semantics for each requirement. An example of this can be seen in Figure 4, which shows the time interval where the requirement should hold and the requirements triggering and stopping condition (if they exist). Both the temporal logic and diagramatic representations of the requirements are useful for understanding and sanity-checking the semantics of the requirement that has been written.

Our experience is that fret and its in-built language, , provide a useful intermediate language to enable formalists and requirements engineers to work in collaboration towards formalising a system’s requirements, without a steep learning curve for either team. The requirements provide a traceability link between natural-language and formalised requirements. fret can generate output based on the requirements that is useful both directly and indirectly for verification. We describe the details of our workflow, beginning with using fret in Sect. 3.

2.2 CoCoSim and Simulink

fret integrates particularly closely with both Simulink and CoCoSpec [8], the latter being a language for describing assume-guarantee contracts using Simulink blocks. fret can be used to generate CoCoSpec contracts for each requirement, which can then be added to the Simulink diagram of the system that the requirements relate to. These contracts are checked during simulations of the diagram in Simulink using the Kind2 model checker [9]. In our methodology, presented in Sect. 3.2, we refer to this as the ‘fret-Supported’ toolchain.

Where Simulink is not used to design the system, or even where it is but other verification methods are desired, the temporal logic version of the requirements can be incorporated into other formal verification approaches. Where a formal verification tool directly accepts temporal logic, this will be straightforward. Otherwise, the temporal logic and versions of the requirements can be used as a guide for formalising the properties and system in other formalisms. In Sect. 3.2 we refer to this as the ‘fret-Guided’ toolchain.

3 Methodology

In this section we describe our methodology and our work to-date on the formalisation and verification of the requirements of our use case. First, we describe our use case, which is an aircraft engine controller. Then, we describe our methodology and how we have applied it to the requirements of the aircraft engine controller use case.

3.1 Use Case: Aircraft Engine Controller

Our example application is a software controller for a high-bypass civilian aircraft turbofan engine, which was provided by our industrial partner on the VALU3S project111The VALU3S project: https://valu3s.eu/. The controller is a representative example of a Full Authority Digital Engine Control (FADEC) system, which monitors and controls everything about the engine, including thrust control, fuel control, power management, health monitoring of the engine, thrust reverser control, and so on.

The controller’s high-level objectives are that it should continue to control the engine (for example keeping settling time, overshoot, and steady state error within acceptable limits), respecting certain operating limits (for example respecting the engine shaft’s upper speed limit), in the presence of:

  • sensor faults (a sensor value deviating more than a given amount from its nominal value, or being unavailable),

  • perturbation of system parameters (a system parameter deviating more than a given amount from its nominal value), and

  • other low-probability hazards (for example, abrupt changes in the outside air pressure).

The controller is also required to detect faults and recover from them, which it does through a change of mode – for example detecting engine surge or stall and changing modes to prevent these hazardous situations. The software controller supplied by our industrial partners is related to existing controllers previously described in the literature [32][35].

ID Description
UC5_R_1 Under sensor faults, while tracking pilot commands, control objectives shall be satisfied (e.g., settling time, overshoot, and steady state error will be within predefined, acceptable limits)
UC5_R_2 Under sensor faults, during regulation of nominal system operation (no change in pilot input), control objectives shall be satisfied (e.g., settling time, overshoot, and steady state error will be within predefined, acceptable limits)
Table 1: The first two requirements for the aircraft engine controller, provided by our industrial partner.

Our industrial partner has supplied 14 English-language requirements and 20 test cases, which provide more detail about the controller’s required behaviour. Table 1 shows the first two requirements for the aircraft engine controller, provided by our industrial partner. As described in Sect. 3.2.1, in addition to the requirements and test cases, we iteratively formalised the requirements through elicitation meetings with our industrial partner. The controller’s software has been designed in Simulink222Simulink: https://mathworks.com/products/simulink.html. We focus on verifying that the high-level design for the controller obeys the requirements.

3.2 Our Three-Phase Methodology

Our workflow takes requirements in natural-language and a Simulink diagram as input, and enables the formal verification of the system’s design against the requirements. At a high level, our approach is split into three distinct phases, shown in Figure 2. First we elicit and formalise the natural-language requirements using fret, in Phase 1. Then we move on to formal verification either supported or guided by fret (Phase 2A and 2B). The ‘fret-Supported’ toolchain uses fret’s built-in translation function to produce contracts that can be incorporated into a Simulink diagram. The ‘fret-Guided’ toolchain uses the formalised requirements to drive the (manual) translation into other formal methods as chosen by the verifier. Both verification phases can be applied in parallel. Finally, Phase 3 involves the assembly of a verification report to capture the verification results and traceability of requirements.

Figure 2: High-Level Flowchart of our Methodology. After Phase 1 is complete, Phases 2A and 2B can occur in parallel. Phases 2A and 2B can both highlight deficiencies in the requirements, prompting a return to Phase 1.

As outlined in Sect. 1, our methodology ‘glues’ the requirements engineering phase and formal verification phases together. These phases use different skills and would often be performed by different teams (or people). With this in mind, our methodology keeps the phases separate, but enables clear communication about the requirements and their formalisation using as a common language. This reduces the likelihood of incorrect or inaccurate requirements being formalised, while side-stepping the need for requirements engineers to learn a fully formal language.

In the remainder of this section we describe the workflow in detail. We make use of the aircraft engine controller, described earlier, as a running example.

3.2.1 Phase 1: Eliciting and Formalising Requirements

Eliciting the system’s requirements and formalising them is a task best completed by a collaboration between those knowledgeable about the system and those knowledgeable about formalisation. Using a formal language during this process can often slow down progress, because the systems engineers must learn the intricacies of the formal method and the formalists often try to formalise too much detail too quickly. Thus, in this phase we take the natural-language requirements as input, and produce the version of the requirements and the ltl specifications of the requirements as output. Using as an intermediate language between the natural- and formal-language versions of the requirements, frees up the requirements elicitation process, but still provides enough formalisation in the requirements to both reduce ambiguities and make the requirements easier to formalise later on.

Figure 3: Flowchart of Phase 1: Eliciting and Formalising Requirements. The circular nodes are start and end points, the rectangular nodes are processes (actions), and the rhomboid nodes are inputs or outputs.

The requirements elicitation phase will be slightly different for each system and set of requirements, but in our workflow (shown in Figure 3) we have separated this phase into a cycle of:

Step 1 (Requirements Elicitation):

An informal process of discussing the natural-language requirements to clarify and reduce ambiguity, it also may uncover new requirements;

Step 2 (Requirements Formalisation):

A manual process of converting the natural-language requirements into .

After Step 2, if the requirements have enough detail then the phase is over. If not, then the workflow cycles back to Step 1.

In our ongoing work, with our industrial partner, we have translated the 14 natural language requirements of the aircraft engine controller into . As an example of how this phase works, we will examine the first two requirements in our case study. Full details describing the requirements eliciation and formalisation process for this use case can be found in our report333https://drive.google.com/drive/folders/1_5SUeh6B9jHrrEyKWN489-xA2zdCp6qx?usp=sharing.

Requirement UC5_R_1 reads: “Under sensor faults, while tracking pilot commands, control objectives shall be satisfied (e.g. settling time, overshoot, and steady state error will be within predefined, acceptable limits)”, which is expressed in as:

((sensorfaults) & (trackingPilotCommands)) Controller shall (controlObjectives)

Similarly, Requirement UC5_R_2 reads: “Under sensor faults, during regulation of nominal system operation (no change in pilot input), control objectives shall be satisfied (e.g. settling time, overshoot, and steady state error will be within predefined, acceptable limits)", which is expressed in as:

(sensorfaults) & (!trackingPilotCommands) Controller shall (controlObjectives)

Through the iterations of requirements elicitation, we added more detail to the requirements, checking with our industrial partners that the requirements were correct. We added detail to the requirements through information gathered from the 20 test cases provided for the use case. For example, Test Case 1 (for Requirement UC5_R_1):

Preconditions: Aircraft is in operating mode and sensor value deviates at most from nominal value Input conditions/steps: Observed aircraft thrust is at value and pilot input changes from to Expected results: Observed aircraft thrust changes and settles to value , respecting control objectives (settling time, overshoot, steady state error)

was split into three child requirements in . As an example, UC5_R_1.1 is the child requirement (of UC5_R_1) that adds details for the Control Objective settling time is written as:

when (diff(r(i),y(i)) > E) if ((sensorValue(S) > nominalValue + R)

(sensorValue(S) < nominalValue - R) (sensorValue(S) = null)

& (pilotInput => setThrust = V2) & (observedThrust = V1) )

Controller shall until (diff(r(i),y(i)) < e)

(settlingTime >= 0) & (settlingTime <= settlingTimeMax)

& (observedThrust = V2)

Figure 4 shows fret’s ‘Update Requirement’ dialogue for UC5_R_1.1. The “Requirement Description” section (bottom left) shows the version of the requirement that we described above. On the right-hand side, the semantics diagram (and accompanying textual description) displays when the requirement becomes active and what its triggering (TC) and stopping (SC) conditions are.

Figure 4: The ‘Update Requirement’ Dialogue Box in fret for Requirement UC5_R_1.1, showing the requirement’s ID, the parent requirement ID, the project name, the requirement’s description in as well as both the LTL representation of the requirements and the diagrammatic semantics for the requirement (on the right-hand panel) showing when the requirement becomes active and what its triggering (TC) and stopping (SC) conditions are.

Once the requirements elicitation and formalisation phase is complete, the requirements can be translated into the input language for the formal method(s) chosen for the verification phase. Depending on the formal method chosen, this translation could be automatic or could be provided by the manual application of a set of systematic translation rules. If the design is described in a Simulink diagram, existing tool-support can take formalised requirements and produce contracts compatible with Simulink. Where the design has not been described using Simulink, or where the verification efforts would benefit from other formal approaches, we can build a formal model of the system from the available descriptions (the requirements, other models, documentation, etc.) and check the properties (formal encodings of the requirements) against the model.

3.2.2 Phase 2A: FRET-Supported Verification

In this phase, we take the requirements and the Simulink design as input, and produce CoCoSpec contracts and verification results as output. The CoCoSpec contracts are built from Simulink blocks so that they can be embedded directly in a Simulink diagram. Because this phase is supported by fret, much of the process is automated. However, the user still needs to provide information to link the requirements to the Simulink design, and the CoCoSpec contracts have to be manually added to the Simulink diagram for verification with the Kind2 model checker.

Figure 5: Flowchart of Phase 2A: Verification supported by fret’s link to CoCoSim. The circular nodes are start and end points, the rectangular nodes are processes (actions), the diamond nodes are decisions, and the rhomboid nodes are inputs or outputs. The double-edged rectangular node (bottom left) represents an external flowchart, in this case Phase 1.

This phase of our workflow has four steps, which can be seen in Figure 5 and which we describe below.

Step 1 (Attach Simulink Model):

fret provides a Matlab script that extracts information from a Simulink diagram in a format that fret can consume. Adding this to the fret project makes it easier for the user to generate the verification conditions in the next step.

Step 2 (Generate Verification Conditions):

In fret, the user can add detail to link the requirements to the Simulink diagram of the system. For example, they can map a component mentioned in the requirements to a block or collection of blocks, or map a variable in the requirements to a signal in the Simulink diagram. Once this is complete, fret can generate CoCoSpec contracts for the Simulink diagram, that represent the requirements. As an example, requirement UC5_R_1 (see Table 1) is translated in the the CoCoSpec contract shown below:

guarantee "UC5_R_1" ((H(not(((sensorfaults) and (trackingPilotCommands))))) or
(not(SI(((((sensorfaults) and (trackingPilotCommands))) and
((pre(not(((sensorfaults) and (trackingPilotCommands))))) or FTP)),
(not((controlObjectives)))))));

This CoCoSpec contract uses the operators: H, SI, pre and FTP. Here H represents the ‘Historically’ past-time ltl operator and SI represents the ‘Since Inclusive’ operator. Then, pre is used to refer to the previous value of the variable following it and FTP refers to the First Time Point of the trace. The specific details of the CoCoSpec generation process are out of the scope of this work but we refer the interested reader to [29] for the full details.

Step 3 (Formal Verification):

Once the CoCoSpec contracts are incorporated into the Simulink diagram, they can be used to verify that the diagram (and so, the design of the system) obeys the contracts (and so, the requirements) during simulation runs using the Kind2 model checker.

Step 4 (Evaluate Results):

If the verification fails, then it is possible that the Simulink diagram is incorrect, so it should be improved until the verification passes. It is also possible that the requirements are incorrect, in which case the workflow returns to Phase 1: Requirements Elicitation and Formalisation. If the verification passes, then the Simulink diagram obeys the requirements.

3.2.3 Phase 2B: FRET-Guided Verification

In this phase we take both the natural-language and versions of the requirements as input, and produce formal properties, a formal model of the system, and verification results as output. This phase uses the requirements to drive formal modelling of the system.

Figure 6: Flowchart of Phase 2B: Verification guide by requirements. The circular nodes are start and end points, the rectangular nodes are processes (actions), the diamond nodes are decisions, and the rhomboid nodes are inputs or outputs. The double-edged rectangular node (bottom left) represents an external flowchart, in this case Phase 1.

We anticipate using formal methods that plug gaps in the verification approach used in Phase 2A. For example, methods that enable more detailed modelling of data, such as Event-B [1]; or more detailed modelling of behaviour, such as csp [19]. Again, this phase occurs in four steps, which can be seen in Figure 6 and we describe each of these individual phases below.

Step 1 (Translate to Verification Conditions):

Using the requirements as the source, produce the properties to be verified in the chosen formal language. This is likely to be a non-formal translation process; but if the chosen formal method’s input language is close to , then the translation will be straightforward. Obviously if the chosen formal method uses temporal logic, then the ltl translations that fret automatically produces could be used directly.

Step 2 (Build Formal System Model):

Since this phase does not directly use a Simulink model, a formal model of the system or design must be constructed separately. This step is likely to involve using the and natural-language requirements, the Simulink model (if it exists), and other sources to ensure that the system model is valid.

Step 3 (Formal Verification):

Using the appropriate tool (such as a model checker or theorem prover) perform the formal verification of the system model (Step 2) against the formal properties (Step 1).

Step 4 (Evaluate Results):

As with the ‘fret-Supported’ toolchain in Phase 2A if the verification fails, then it is possible that the formal system model is incorrect and requires updates to make it pass the verification. It is also possible that the requirements themselves are wrong, and so the workflow returns to Phase 1: Requirements Elicitation and Formalisation. If the verification passes, then the system model obeys the formalised requirements.

3.2.4 Phase 3: Verification Report

Using the verification results returned by Phase 2A and 2B, along with the formalised requirements produced as part of Phase 1, a verification report is assembled. This report could take various forms, depending on what is required of the development process by laws, regulations, etc. Ensuring that verification (particularly formal verification [26]) can provide evidence of software’s safety that is understandable by, and acceptable to, a regulator (or other organisation with supervisory responsibilities) is key in safety-critical domains.

The report could be a natural-language report, structured as a safety case in a notation like the Goal Structuring Notation [20] or using tools such as the Assurance Case Automation Toolset (AdvoCATE) [11], or a combination of these formats. The key point is that the report facilitates traceability of the verified requirements and explains the extent of the verification activities, which may be useful during the assurance process.

4 Discussion

Our methodology, as presented in this paper, unifies and facilitates the use of multiple verification approaches whilst maintaining and supporting requirements traceability. We demonstrate our methodology through an industrial use case of an aircraft engine controller. Our methodology illustrates the benefits of using a formal approach to requirements capture so that formalised requirements can be directly fed into the verification process with minimal effort.

Simulink is used extensively in the aerospace domain to model systems. We use FRET for requirements formalisation and elicitation because it has direct links to tools used for verifying Simulink diagrams. However, it has been shown that these tools are not always sufficiently scalable for complex requirements [7] so we also outline a pathway where other formal methods can be incorporated to support verification.

Our methodology provides a way to build and examine a series of prototype systems/models before concrete implementation occurs. In particular, we can consider the fret formalised requirements as a prototype formal model. They contain the conditions that the formal model must obey and some of the requirements even define how certain variables are updated/how the system should behave in a given context. Then, we can analyse the Simulink model which is a prototype implementation that can be examined with respect to the generated CoCoSpec contracts. In addition, we can build formal models of the system which can be treated as prototype implementations since they mirror the behaviour of the final system, with some details abstracted.

4.1 Evolution of Requirements

Our high-level methodology, as captured in Figure 2, contains optional loops that go back from the verification phases (Phases 2A and 2B) to the requirements elicitation phase (Phase 1). The option to return to this earlier phase enables the methodology to respond to errors that are only identified in Phase 2A or 2B, but indicate incorrect or inaccurate requirements. It is important that the requirements are captured as accurately as possible; if errors or omissions in the requirements are found, then any changes to remedy these problems may also propagate to the accompanying Simulink diagram(s) or formal model(s). However, such changes would also propagate if a non-formal requirements engineering approach were being used.

fret provides some assistance with changes propagating from updated requirements to their derived models and diagrams. First, when a requirement is updated, fret automatically converts it to ltl, so there is no user overhead in this step. The updated requirement can be re-exported to CoCoSpec (Phase 2A), or the updated requirement can be re-translated into the formal language used before (Phase 2B). These are manual steps, but since the contracts or formal properties have been built from each of the requirements, the traceability provided by fret makes their regeneration more straightforward than if they had been built directly from the natural language requirements.

In addition to revision of the verification conditions, revision of the system’s design and associated models may be needed if the requirements are found to be wrong. Again, this would be a manual process of updating the design to match the changes to the requirements, but this would also be required in a non-formal approach. In practice, software development follows an iterative process where additional requirements may be added or existing ones modified as development continues and errors or inconsistencies are revealed in the design. Here, fret provides traceability features that make these updates easier to implement. Requirement IDs enable links between requirements to be traced, the source of the requirement (for example its natural-language counterpart) can be captured in the rationale field, and the CoCoSpec contracts that fret automatically produces share the name of the requirement from which they were generated. These features mean that the requirements can be linked back to the natural-language requirements, and linked forward to the verification conditions and system designs, therefore finding the target for updates is relatively straight-forward.

4.2 fret-Supported vs fret-Guided Verification

As previously mentioned, Phase 2A (fret-Supported Verification) and Phase 2B (fret-Guided Verification) can be applied separately or together in parallel. Both phases apply formal verification to the system’s design, but using different languages to specify the properties and using different ways of describing the design itself. Here we discuss the interactions between these two phases, the situations where each might be useful, and the benefits of applying both phases to a system’s design.

The main difference between the two verification phases are in the targets of the verification. Phase 2A targets a Simulink diagram, using the Kind2 model checker. In contrast, Phase 2B targets a separate formal model of the system, built in a formal method of choice. Direct use of a Simulink diagram as the description of the system is useful (if the diagram exists), but the available checks are limited to those directly describable in . Introducing another formal language, and its associated tools, might be advantageous. Using the approach supported in Phase 2B may enable the verification of properties that it was not possible to verify using the approach supported in Phase 2A, as was found to be the case in related work [7].

Using both phases in parallel provides a more corroborative approach to verification, where different styles of verification are used to corroborate each other [36]. Corroborative verification is often focussed on applying multiple verification techniques to the same set of properties, but applying both phases in our methodology also offers the developer the chance to pick and choose which techniques to use to verify which specific requirements. In this ‘divide and conquer’ strategy, a formal method can be chosen that complements the CoCoSpec approach and plugs any gaps or limitations that it may have, as was the approach in [7].

Further, using both phases produces two independent designs, both derived from the same set of formalised requirements. Comparing the design in Simulink to a design in a formal language may help to identify inconsistencies or ambiguities in either design, or in the requirements themself, that were not identified during requirements elicitation. Exploring different designs in this way may produce more robust implementations, because the requirements have been viewed through a variety of lenses.

5 Related Work

In this section we describe some of the related work in the literature, focussing on the themes of requirement specification, the formal verification of aircraft systems, and the formal verification of Simulink diagrams.

5.1 Requirements Specification

Specifying verifiable and precise requirements can often delay the verification process [34]. Requirements elicitation and formalisation tools can aid this process by helping the user to define unambiguous requirements from the outset. In safety- or mission-critical domains such as the aerospace sector, development is often driven by high-level requirements that are gradually decomposed as the project continues.

Some approaches to requirements engineering simply use natural-language. For example, IBM’s Dynamic Object-Oriented Requirements System (DOORS)444https://www.ibm.com/docs/en/ermd/9.7.0?topic=overview-doors is frequently used in industry to manage and support traceability of requirements. In DOORS, requirements are represented in natural-language without any view to formalise them or reduce ambiguities so that they can be easily used by formal methods.

In contrast, there have been multiple approaches developed that support the use of more formal, logical requirements. These include fret, which uses a structured natural language to provide an ltl semantics for formalised requirements [16]. fret uses the Structured Assertion Language for Temporal Logic (SALT), which is a general purpose assertion language with property pattern features as an intermediate language [5].

Formalised requirements have also been added to existing, non-formal approaches. For example, the Easy Approach to Requirements Syntax (EARS) uses informal templates to define high-level requirements [28]. However, recent work has provided a formalisation for these templates in ltl [23]. Despite this advancement, EARS does not support contract generation whereas fret does.

Formal Verification of Aircraft Systems

Formal verification tools and techniques have been applied to an array aircraft systems. This inlcudes the specification and verification of a landing gear system in Event-B [27] and Abstract State Machines [21]. Related work focuses on the verification of a sensor voting module used in the landing gear system using model-checking with Asmeta [3].

An aadl Model of an F-16 Autopilot Controller has been translated into Stateful Timed csp, which was then model checked in the Process Analysis Toolkit (PAT) checker. The Autopilot Controller is a non-linear model that simulates the dynamics of the real aircraft. This work focuses on verifying safety, liveness, and trace refinement properties [38]. Other work that incorporates csp includes the use of Hybrid csp to model and verify an aircraft system. The example system is a controller for correcting an aircraft’s position when it has deviated from its intended flight path [31].

In [4], Backes et al. use the Assume Guarantee REasoning Environment (AGREE) [10] for compositional verification of an aadl model of a Quad-Redundant Flight Control System. They use the Kind2 and JKind model-checkers for verification. One of the main difficulties that they had was in formalising natural language requirements at a level of abstraction and detail that was amenable to compositional verification. Our work seeks to streamline this process by including detailed requirements formalisation from the outset.

Formal Verification of Simulink Diagrams

The popularity of Simulink for designing systems has lead to a variety of formal verification techniques that target Simulink diagrams. In addition to its own Simulink Design Verifier [17], the literature contains examples both of model checking and theorem proving as the formal verification approach.

Some of the approaches in the literature translate the elements of a Simulink diagram into the input language of a theorem prover. For example, Herber et al. 2013 [18] present an automatic translation from discrete-time Simulink diagrams to the input language of the UCLID system verification toolkit555UCLID: https://github.com/uclid-org/uclid. Similarly, Reicherdt and Glensner 2014 [33] translate discrete-time Simulink diagrams into Boogie, an intermediate language for the Z3 verification framework666Boogie: https://www.microsoft.com/en-us/research/project/boogie-an-intermediate-verification-language/?from=http%3A%2F%2Fresearch.microsoft.com%2Fen-us%2Fprojects%2Fboogie%2F.

Meenakshi et al. [30] present a tool that automatically translates discrete-time Simulink diagrams into the input language of the model checker NuSMV777NuSMV: https://nusmv.fbk.eu/index.html. Their tool parses information about the Simulink diagram from its textual representation in an mdl file, then translates it for NuSMV. The restriction to discrete-time diagrams is due to NuSMV only catering for discrete models. Their tool can also reverse the translation, to aid with debugging the Simulink diagram from counterexamples.

To capture the continuous-time elements of Simulink diagrams, Liebrenz et al. 2018 [22] provide an automatic translation to dl to enable interactive theorem proving using KeYmaera X888KeYMaera X: https://www.keymaerax.org/index.html . However, their automatic translation assumes: no algebraic loops, no S Function blocks999S (System) Function blocks wrap a program, written in either: MATLAB, C C++, or FORTRAN code, and no external scripts or libraries.

Instead of translating Simulink diagrams into another language, Bernardeschi et al. 2018 [6] use co-simulation, where Simulink diagrams are simulated in parallel with a model in the PVS theorem prover101010PVS: http://pvs.csl.sri.com/.. The PVS model captures the discrete semantics of the system and the Simulink model captures the continuous dynamics of the system’s environment. They use PVSio-Web111111PVSio-Web: http://www.pvsioweb.org/, which is a Web framework that enables formal-model-based development of Human-Machine Interfaces, to connect the PVS model to the Simulink diagram. They also use an additional block in the Simulink diagram to extract the simulation data. PVSio-Web uses this block’s output to keep the PVS model in-sync with the Simulink simulation.

The addition of blocks to the Simulink diagram is the approach taken by CoCoSpec [8], which we use in our methodology. This approach has already been described in the literature, for example In Araiza-Illan et al. 2014 [2] where assertion blocks are added to Simulink diagrams to describe the properties (over the Simulink signals) that the system should preserve. These assertions take the form of a require block, which is used to describe a Hoare triple: . A require block is constructed from an Enabled Subsystem, which conditionally executes when a control signal (the precondition) is positive and itself contains an Assert block. Once this specification is added to the Simulink diagram, they provide an automatic translation into the input language of the Why3121212Why3: http://why3.lri.fr/ theorem proving platform, and used CVC3131313CVC3: https://cs.nyu.edu/acsys/cvc3/ to verify the Simulink diagram against the properties described by the assertion blocks. Their translation strategy works on Simulink signals that are scalar and discrete, but they aimed to extend the approach to continuous time.

6 Conclusions and Future Work

This paper has presented our methodology for developing a verifiable system using a formal requirements-driven approach. This work is targetted for the aerospace domain and we illustrate our methodology via an aircraft engine controller use case that has been supplied by our industry partner, United Technologies Research Center in Ireland.

Our methodology is rooted in requirements elicitation and formalisation, to drive the derivation of appropriate verification conditions and support traceability. We have chosen fret as the basis for this approach because it has support for verifying Simulink diagrams and previous work has shown that fret requirements are relatively straightforward to incorporate into other verification formalisms [7]. In saying that, future work will seek to make this link between fret and other formal methods more seamless.

Our methodology provides an avenue to harness the power of formal methods in a way that complements existing verification techniques, which streamlines the process of developing verifiable aircraft engine controllers. Further, the requirements-driven approach that we adopt supports the traceability of requirements throughout the verification process. The combination of these existing tools applied to an industrial aircraft engine controller has, to the best of our knowledge, not been explored before. We also believe that our approach is flexible and, although we focus on a particular suite of tools, that other tools may be easily incorporated where necessary.

We are continuing to elaborate this use case alongside our industry partner whose primary focus is the aerospace domain. However, we also intend to investigate other domains which make extensive use of Simulink and related approaches to examine and evaluate our work more broadly.

References

  • [1] Jean-Raymond Abrial. Modeling in Event-B: system and software engineering. Cambridge University Press, 2010.
  • [2] Dejanira Araiza-Illan, Kerstin Eder, and Arthur Richards. Formal verification of control systems’ properties with theorem proving. In 2014 UKACC International Conference on Control (CONTROL), pages 244–249, Loughborough, UK, July 2014. IEEE. URL: http://ieeexplore.ieee.org/document/6915147/, doi:10.1109/control.2014.6915147.
  • [3] Paolo Arcaini, Angelo Gargantini, and Elvinia Riccobene. Offline model-based testing and runtime monitoring of the sensor voting module. In Communications in Computer and Information Science, pages 95–109. Springer International Publishing, 2014. URL: https://doi.org/10.1007%2F978-3-319-07512-9_7, doi:10.1007/978-3-319-07512-9_7.
  • [4] John Backes, Darren Cofer, Steven Miller, and Michael W Whalen. Requirements analysis of a quad-redundant flight control system. In NASA Formal Methods Symposium, pages 82–96. Springer, 2015.
  • [5] Andreas Bauer, Martin Leucker, and Jonathan Streit. Salt—structured assertion language for temporal logic. In International Conference on Formal Engineering Methods, pages 757–775. Springer, 2006.
  • [6] Cinzia Bernardeschi, Andrea Domenici, and Paolo Masci. A PVS-Simulink Integrated Environment for Model-Based Analysis of Cyber-Physical Systems. IEEE Transactions on Software Engineering, 44(6):512–533, June 2018. URL: https://ieeexplore.ieee.org/document/7900400/, doi:10.1109/tse.2017.2694423.
  • [7] Hamza Bourbouh, Marie Farrell, Anastasia Mavridou, Irfan Sljivo, Guillaume Brat, Louise A. Dennis, and Michael Fisher. Integrating Formal Verification and Assurance: An Inspection Rover Case Study. In Mariano M. Moscato, Aaron Dutle, Laura Titolo, Ivan Perez, and César A. Muñoz, editors, NASA Formal Methods, volume 12673, pages 53–71. Springer International Publishing, 2021. Series Title: Lecture Notes in Computer Science. URL: https://link.springer.com/10.1007/978-3-030-76384-8_4, doi:10.1007/978-3-030-76384-8_4.
  • [8] Hamza Bourbouh, Pierre-Loïc Garoche, Thomas Loquen, Éric Noulard, and Claire Pagetti. Cocosim, a code generation framework for control/command applications an overview of cocosim for multi-periodic discrete simulink models. In 10th European Congress on Embedded Real Time Software and Systems (ERTS 2020), 2020.
  • [9] Adrien Champion, Alain Mebsout, Christoph Sticksel, and Cesare Tinelli. The kind 2 model checker. In International Conference on Computer Aided Verification, pages 510–517. Springer, 2016.
  • [10] Darren Cofer, Andrew Gacek, Steven Miller, Michael W Whalen, Brian LaValley, and Lui Sha. Compositional verification of architectural models. In NASA Formal Methods Symposium, pages 126–140. Springer, 2012.
  • [11] Ewen Denney, Ganesh Pai, and Josef Pohl. Advocate: An assurance case automation toolset. In International Conference on Computer Safety, Reliability, and Security, pages 8–21. Springer, 2012.
  • [12] Marie Farrell, Matt Luckcuck, and Michael Fisher. Robotics and Integrated Formal Methods: Necessity Meets Opportunity. In C Furia and K Winter, editors, Integrated Formal Methods, volume 11023 of Lecture Notes in Computer Science, pages 161–171, Cham, 2018. Springer. tex.ids: farrellRoboticsIntegratedFormal2018. URL: http://arxiv.org/abs/1805.11996https://doi.org/10.1007/978-3-319-98938-9_10, doi:10.1007/978-3-319-98938-9_10.
  • [13] Michael Fisher. An introduction to practical formal methods using temporal logic. John Wiley & Sons, 2011.
  • [14] Michael Fisher, Rafael C. Cardoso, Emily C. Collins, Christopher Dadswell, Louise A. Dennis, Clare Dixon, Marie Farrell, Angelo Ferrando, Xiaowei Huang, Mike Jump, Georgios Kourtis, Alexei Lisitsa, Matt Luckcuck, Shan Luo, Vincent Page, Fabio Papacchini, and Matt Webster. An Overview of Verification and Validation Challenges for Inspection Robots. Robotics, 10(2):67, April 2021. URL: https://www.mdpi.com/2218-6581/10/2/67, doi:10/gjvnqn.
  • [15] Dimitra Giannakopoulou, Anastasia Mavridou, Julian Rhein, Thomas Pressburger, Johann Schumann, and Nija Shi. Formal Requirements Elicitation with FRET. Pisa, March 2020. NTRS Author Affiliations: NASA Ames Research Center, Stinger Ghaffarian Technologies Inc. (SGT Inc.), Technische Univ. NTRS Report/Patent Number: ARC-E-DAA-TN77785 NTRS Document ID: 20200001989 NTRS Research Center: Ames Research Center (ARC). URL: https://ntrs.nasa.gov/citations/20200001989.
  • [16] Dimitra Giannakopoulou, Thomas Pressburger, Anastasia Mavridou, and Johann Schumann. Generation of formal requirements from structured natural language. In International Working Conference on Requirements Engineering: Foundation for Software Quality, pages 19–35. Springer, 2020.
  • [17] Grégoire Hamon, Bruno Dutertre, Levent Erkok, John Matthews, Daniel Sheridan, David Cok, John Rushby, Peter Bokor, Sandeep Shukla, Andras Pataricza, et al. Simulink design verifier-applying automated formal methods to simulink and stateflow. In Third Workshop on Automated Formal Methods, 2008.
  • [18] Paula Herber, Robert Reicherdt, and Patrick Bittner. Bit-precise formal verification of discrete-time MATLAB/Simulink Models using SMT Solving. In 2013 Proceedings of the International Conference on Embedded Software (EMSOFT), pages 1–10, Montreal, QC, Canada, September 2013. IEEE. URL: http://ieeexplore.ieee.org/document/6658586/, doi:10.1109/emsoft.2013.6658586.
  • [19] Charles Antony Richard Hoare. Communicating sequential processes. Communications of the ACM, 21(8):666–677, 1978.
  • [20] Tim Kelly and Rob Weaver. The Goal Structuring Notation – A Safety Argument Notation. In Dependable Systems and Networks 2004 Workshop on Assurance Cases, 2004. URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.66.5597&rep=rep1&type=pdf.
  • [21] Felix Kossak. Landing gear system: An asm-based solution for the abz case study. In Frédéric Boniol, Virginie Wiels, Yamine Ait Ameur, and Klaus-Dieter Schewe, editors, ABZ 2014: The Landing Gear Case Study, pages 142–147, Cham, 2014. Springer International Publishing.
  • [22] Timm Liebrenz, Paula Herber, and Sabine Glesner. Deductive Verification of Hybrid Control Systems Modeled in Simulink with KeYmaera X. In Jing Sun and Meng Sun, editors, Formal Methods and Software Engineering, volume 11232, pages 89–105, Cham, 2018. Springer International Publishing. Series Title: Lecture Notes in Computer Science. URL: http://link.springer.com/10.1007/978-3-030-02450-5_6, doi:10.1007/978-3-030-02450-5_6.
  • [23] Levi Lúcio, Salman Rahman, Chih-Hong Cheng, and Alistair Mavin. Just formal enough? automated analysis of ears requirements. In NASA Formal Methods Symposium, pages 427–434. Springer, 2017.
  • [24] Matt Luckcuck. Using formal methods for autonomous systems: Five recipes for formal verification. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, July 2021. URL: https://arxiv.org/abs/2012.00856, doi:10.1177/1748006X211034970.
  • [25] Matt Luckcuck, Marie Farrell, Louise A. Dennis, Clare Dixon, and Michael Fisher. Formal Specification and Verification of Autonomous Robotic Systems: A Survey. ACM Computing Surveys, 52(5):1–41, October 2019. URL: https://dl.acm.org/doi/10.1145/3342355, doi:10.1145/3342355.
  • [26] Matt Luckcuck, Marie Farrell, Louise A. Dennis, Clare Dixon, and Michael Fisher. A Summary of Formal Specification and Verification of Autonomous Robotic Systems. In Wolfgang Ahrendt and Silvia Lizeth Tapia Tarifa, editors, Integrated Formal Methods, volume 11918, pages 538–541, Cham, 2019. Springer International Publishing. Series Title: Lecture Notes in Computer Science. URL: http://link.springer.com/10.1007/978-3-030-34968-4_33, doi:10.1007/978-3-030-34968-4_33.
  • [27] Amel Mammar and Régine Laleau. Modeling a landing gear system in event-b. International Journal on Software Tools for Technology Transfer, 19(2):167–186, 2017.
  • [28] Alistair Mavin. Listen, then use ears. IEEE software, 29(2):17–18, 2012.
  • [29] Anastasia Mavridou, Hamza Bourbouh, Pierre-Loic Garoche, Dimitra Giannakopoulou, Tom Pressburger, and Johann Schumann. Bridging the gap between requirements and model analysis: Evaluation on ten cyber-physical challenge problems. 2020.
  • [30] B. Meenakshi, Abhishek Bhatnagar, and Sudeepa Roy. Tool for Translating Simulink Models into Input Language of a Model Checker. In Formal Methods and Software Engineering, volume 4260, pages 606–620, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. Series Title: Lecture Notes in Computer Science. URL: http://link.springer.com/10.1007/11901433_33, doi:10.1007/11901433_33.
  • [31] Yu Peng, Shuling Wang, Naijun Zhan, and Lijun Zhang. Extending hybrid CSP with probability and stochasticity. In Dependable Software Engineering: Theories, Tools, and Applications, pages 87–102. Springer International Publishing, 2015. URL: https://doi.org/10.1007%2F978-3-319-25942-0_6, doi:10.1007/978-3-319-25942-0_6.
  • [32] I Postlethwaite, R Samar, BW Choi, and DW Gu. A Digital Mmultimode H Controller for the Spey Turbofan Engine. In Proceedings of the European Control Conference, 1995.
  • [33] Robert Reicherdt and Sabine Glesner. Formal Verification of Discrete-Time MATLAB/Simulink Models Using Boogie. In Software Engineering and Formal Methods, volume 8702, pages 190–204, Cham, 2014. Springer International Publishing. Series Title: Lecture Notes in Computer Science. URL: http://link.springer.com/10.1007/978-3-319-10431-7_14, doi:10.1007/978-3-319-10431-7_14.
  • [34] Kristin Yvonne Rozier. Specification: The Biggest Bottleneck in Formal Methods and Autonomy. In Sandrine Blazy and Marsha Chechik, editors, Verified Software. Theories, Tools, and Experiments, Lecture Notes in Computer Science, pages 8–26, Cham, 2016. Springer International Publishing. doi:10.1007/978-3-319-48869-1_2.
  • [35] Raza Samar and Ian Postlethwaite. Design and Implementation of a Digital Multimode H Controller for the Spey Turbofan Engine. Journal of Dynamic Systems, Measurement, and Control, 132(1):011010, January 2010. URL: https://asmedigitalcollection.asme.org/dynamicsystems/article/doi/10.1115/1.4000656/466534/Design-and-Implementation-of-a-Digital-Multimode-H, doi:10.1115/1.4000656.
  • [36] Matt Webster, David Western, Dejanira Araiza-Illan, Clare Dixon, Kerstin Eder, Michael Fisher, and Anthony G Pipe. A corroborative approach to verification and validation of human–robot teams. The International Journal of Robotics Research, 39(1):73–99, January 2020. URL: http://journals.sagepub.com/doi/10.1177/0278364919883338, doi:10.1177/0278364919883338.
  • [37] Jim Woodcock, Peter Gorm Larsen, Juan Bicarregui, and John Fitzgerald. Formal methods: Practice and Experience. ACM Computing Surveys, 41(4):1–36, 2009. URL: http://portal.acm.org/citation.cfm?doid=1592434.1592436, doi:10.1145/1592434.1592436.
  • [38] Feng Zhang, Yongwang Zhao, Dianfu Ma, and Wensheng Niu. Formal verification of behavioral AADL models by stateful timed CSP. IEEE Access, 5:27421–27438, 2017. URL: https://doi.org/10.1109%2Faccess.2017.2770323, doi:10.1109/access.2017.2770323.