Abstract I/O Specification

01/29/2019 ∙ by Willem Penninckx, et al. ∙ 0

We recently proposed an approach for the specification and modular formal verification of the interactive (I/O) behavior of programs, based on an embedding of Petri nets into separation logic. While this approach is scalable and modular in terms of the I/O APIs available to a program, enables composing low-level I/O actions into high-level ones, and enables a convenient verification experience, it does not support high-level I/O actions that involve memory manipulation as well as low-level I/O (such as buffered I/O), or that are in fact "virtual I/O" actions that are implemented purely through memory manipulation. Furthermore, it does not allow rewriting an I/O specification into an equivalent one. In this paper, we propose a refined approach that does have these properties. The essential insight is to fix the set of places of the Petri net to be the set of separation logic assertions, thus making available the full power of separation logic for abstractly stating an arbitrary operation's specification in Petri net form, for composing operations into an I/O specification, and for equivalence reasoning on I/O specifications. Our refinement resolves the issue of the justification of the choice of Petri nets over other formalisms such as general state transition systems, in that it "refines them away" into the more essential constructs of separating conjunction and abstract nested triples. To enable a convenient treatment of input operations, we propose the use of prophecy variables to eliminate their non-determinism. We illustrate the approach through a number of example programs, including one where subroutines specified and verified using I/O specifications run as threads communicating through shared memory. The theory and examples of the paper have been machine-checked using the Iris library for program verification in the Coq proof assistant.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

While great progress has been made in recent decades on approaches for modular formal verification of memory safety of imperative programs, as well as functional correctness of data structures and algorithms, even in the presence of pointer manipulation (or other types of aliasing), coarse-grained or fine-grained concurrency, and higher-order programming, the issue of specifying and modularly verifying the actual observable interactive behavior of the program as a whole, such as achieved through I/O APIs including file I/O, network I/O, graphical user interface APIs, etc., has received much less attention; verification of interactive behavior, if done at all, has mostly been performed at the level of abstract models, often using techniques such as model checking, rather than at the level of source code, integrated with the Hoare logic used for verifying memory safety and data structure correctness. This leaves an unverified gap between the abstract models and the source code.

In this paper, we address this issue by proposing an approach for integrating I/O verification into a Hoare-style modular program verification approach where each function of the program is assigned a specification consisting of a precondition and a postcondition, and then each function is verified against its specification, under the assumption that its callees satisfy theirs. This involves in particular addressing the question of what the specifications of the platform’s I/O API functions, and the program’s main function, should look like, to express the program’s behavioral requirements. Our goal is that the approach should be applicable to an annotation-based verification tool, such as e.g. the VeriFast tool, for programs written in real languages, such as C or Java, against real platform APIs, such as stdio.h or java.io.

As far as we know, the only approach that has been proposed so far to address this goal is the one proposed by Penninckx et al. (2015), where a program’s behavioral requirements are expressed as Petri nets embedded into separation logic (O’Hearn et al., 2001). This approach scales and is modular with respect to the number of I/O APIs available to a program, it allows the program (or program libraries) to define higher-level I/O actions on top of the platform I/O actions (such that a program module can be agnostic as to which actions are primitive and which are composite), and it integrates well into existing separation logic tool support approaches such as symbolic execution with symbolic heaps, yielding a convenient, low-overhead verification experience. However, this approach does not support high-level I/O actions that involve memory manipulation as well as low-level I/O (such as buffered I/O), or that are in fact “virtual I/O” actions that are implemented purely through memory manipulation (such as offered by Java’s ByteArrayOutputStream). Furthermore, it does not allow rewriting an I/O specification into an equivalent one.

In this paper, we propose a refined approach that does have these properties. The essential insight is to fix the set of places of the Petri net to be the set of separation logic assertions, thus making available the full power of separation logic for abstractly stating an arbitrary operation’s specification in Petri net form, for composing operations into an I/O specification, and for equivalence reasoning on I/O specifications. To enable a convenient treatment of non-deterministic input operations, we propose the use of prophecy variables to eliminate the non-determinism.

Our proposed refinement resolves the issue of the justification of the choice of Petri nets over other formalisms, such as general state transition systems, in that while the refined approach can still be seen as applying the Petri nets formalism, it can also be explained straightforwardly without any reference to Petri nets, as simply applying the essential concepts of separating conjunction and an abstract form of nested Hoare triples (e.g. (Schwinghammer et al., 2011; Krebbers et al., 2017)).

We illustrate the approach through a number of example programs, including one where subroutines specified and verified using I/O specifications run as threads communicating through shared memory. The theory and examples of the paper have been machine-checked using the Iris (Krebbers et al., 2017) library for verification of concurrent programs in the Coq proof assistant.

The rest of this paper is structured as follows. In §2, we define the syntax and the semantics of the programming language that we will use to present our approach. In §3, we recall the Petri net-based specification approach (Penninckx et al., 2015) that we refine in this work. In §4, we introduce our refined approach, and we motivate it by means of the example of buffered output. In §5, we illustrate the problem of proving I/O-style specifications for in-memory input operations by means of a chat server example. In §6 we introduce prophecy variables to address this problem. In §7 we extend our programming languge from §2 and our Hoare logic from §3 to support concurrency, which we then use in §8 to verify an implementation of the channels construct used in the chat server against I/O-style specifications. We end the paper with a discussion of related work (§9) and a conclusion (§10).

2. A Programming Language with I/O

2.1. The Programming Language

We present the basic idea of our approach in the context of a simple ML-like programming language with support for I/O. Its grammar is as follows:

We assume a set of program variables and of primitive I/O tags.

We define and where does not appear in . We define and , and where does not appear in or . Furthermore, we define and . We encode characters as tuples of booleans and strings as lists of characters.

To define the language’s semantics, we define the values and the evaluation contexts as follows:

We assume an infinite set of heap locations.

We define the I/O actions ; in , we call the argument and the result. The traces are the lists of I/O actions. We use to denote the empty list and to denote list concatenation.

We define the heaps as the finite partial functions from heap locations to values.

We define the configurations . We define a labeled head reduction relation , a labeled small-step relation , and a labeled reachability relation in Figure 1.

h, cases(inl(v), λx. e, _) ↪ϵ h, e[v/x] h, cases(inr(v), _, λx. e) ↪ϵ h, e[v/x] h, fst((v, v’)) ↪ϵ h, v h, snd((v, v’)) ↪ϵ h, v’ h, (λx. e)(v) ↪ϵ h, e[v/x] h, assert(true) ↪ϵ h, () ℓ∉dom(h) h, ref(v) ↪ϵ h[ℓ:= v], ℓ ℓ∈dom(h) h, !ℓ↪ϵ h, h(ℓ) ℓ∈dom(h) h, ℓ←v ↪ϵ h[ℓ:= v], () h, t(v) ↪t(v, v’) h, v’ h, e ↪τ h, e’ h, K[e/∙] →τ h, K[e’/∙] γ→^*ϵ γ γτ γ
γ’ →^*τ γγ→^*ττ γ

Figure 1. The labeled head reduction relation , the labeled small-step relation , and the labeled reachability relation

We say a configuration is finished if its expression is a value: , and that it has failed if it is not finished and not reducible: .

2.2. I/O Specifications

A foundational way of specifying the desired I/O behavior of a program is in the form of a prefix-closed111A set is prefix-closed if implies . set of traces. We say a configuration satisfies such a specification, denoted , if for any configuration reachable from via a trace (implying that both the program and the environment behave according to ), has not failed and furthermore for any I/O action that can perform, the trace is in , for some :

For example ( denotes that is a prefix of : ):

where in the program . This specification constrains both the program and the environment: it specifies that shall return only booleans, and that the program’s first action, if any, shall be to get a boolean, and its second action, if any, shall be to put its negation, and that it shall not perform any further actions. The program is allowed to get stuck (and it generally does) if returns something other than a boolean.

In this paper, we focus on safety properties only; we do not consider verifying termination or liveness properties. Still, we may wish to express that a program satisfies specification and that furthermore, if it terminates, it shall have performed a trace from set , where is the program’s result. We can encode this by extending the set of I/O tags with an tag and specifying that .

2.3. An Unlabeled Semantics

Most modular program verification approaches proposed in the literature assume an unlabeled operational semantics and simply verify that the program does not reach a failed configuration. Fortunately, we can encode satisfaction of an I/O specification into a statement of this form by using a monitoring semantics, defined by an unlabeled small-step relation over instrumented configurations which include an I/O specification in the form of a prefix-closed set of traces.

For the example programming language, in the monitoring semantics, the step rule for I/O expressions is as follows:

The other step rules do not affect, and are not affected by, the I/O specification.

Lemma 2.1 ().

If and then .

We say a configuration is safe, denoted , if no failed configuration is reachable from it.

Theorem 2.2 ().

If then .

3. Recap of the Petri net approach

In this section, we recall the I/O specification approach presented by Penninckx et al. (2015).222Our presentation differs in unimportant ways from that of Penninckx et al. (2015). In subsequent sections, we propose a number of refinements to this approach, to achieve more abstract I/O specifications.

3.1. Petri nets for I/O specification

A Petri net is defined by a set of places , ranged over by and , and a set of transitions. A marking of a Petri net maps each place to the number of tokens present at that place. Given a marking, a transition can fire if there is a token at each of its pre-places. Firing the transition removes one token from each of the transition’s pre-places and adds one token to each of its post-places.

We use the following notation for markings: ; ; ; .

Petri nets can be used to denote I/O specifications by labeling some transitions with I/O actions. In particular, we will use Petri nets whose transitions are of the following form:

where are the pre-places and are the post-places.

A Petri net, given by its set of transitions, defines a labeled step relation and a corresponding labeled reachability relation on markings: t(p, v, v’, q) ∈N V ⊎{[p]}→t(v, v’) V ⊎{[q]} split(p, q, q’) ∈N V ⊎{[p]}→ϵ V ⊎{[q, q’]} join(p, p’, q) ∈N V ⊎{[p, p’]}→ϵ V ⊎{[q]} noop(p, q) ∈N V ⊎{[p]}→ϵ V ⊎{[q]}

We define . Notice that this set is always prefix-closed.

3.2. A Separation Logic for I/O Verification

We can verify that a program satisfies the I/O specification implied by a marking of a Petri net by means of a Hoare logic (more specifically: a separation logic) whose assertions describe a heap and a marking: .333 denotes the powerset of .

We define and . We define where means .

We define .

We define the meaning of correctness judgments:

where

and

and postconditions are functions from values to assertions. We lift operations on assertions pointwise to operations on postconditions. Also, we usually write postconditions using the notation instead of , where stands for result.

The logic supports only result-deterministic I/O specifications, i.e. ones that do not underspecify the results of I/O actions. However, this is not a significant restriction, since any I/O specification T can be written as the union of a set of result-deterministic I/O specifications, and we have the property . For each , can be verified using the Hoare logic.

We say an assertion precedes an assertion , denoted , if .

From these definitions, we can derive the proof rules shown in Figure 2.

{Q(v)} v {Q} {P} e[v/x] {Q} {P} cases(inl(v), λx. e, _) {Q} {P} e[v/x] {Q} {P} cases(inr(v), _, λx. e) {Q} {P} v_1 {Q} {P} fst((v_1, v_2)) {Q} {P} v_2 {Q} {P} snd((v_1, v_2)) {Q} {P} e[v/x] {Q} {P} (λx. e)(v) {Q} {Trueref(v) {res ↦v} {ℓ↦v} !ℓ {ℓ↦v ∧res = v} {ℓ↦_} ℓ←v {ℓ↦v} {token(p) ∧t(p, v, v’, q) ∈N} t(v) {token(q) ∧res = v’} {P} e {Q}
∀v. {Q(v)} K[v/∙] {Q’} {P} K[e/∙] {Q’} P ⊑P’
{P’} e {Q}
Q ⊑Q’ {P} e {Q’} {P} e {Q} {P * R} e {Q * R} ∀i ∈I. {P_i} e {Q} {_i P_i} e {Q} P ⊆P’ P ⊑P’ P ⊑P’ P * R ⊑P’ * R split(p, q, q’) ∈N token(p) ⊑token(q) * token(q’) join(p, p’, q) ∈N token(p) * token(p’) ⊑token(q) noop(p, q) ∈N token(p) ⊑token(q)

Figure 2. Proof rules of the separation logic for I/O verification

3.3. Examples

The following diagram denotes a Petri net with places , marking , and transitions , where is an I/O tag:

When used as an I/O specification, it allows the program to perform the I/O actions and , once, in that order. Indeed, given the marking shown (one token in place and zero tokens in places and ), the transition labeled can fire (because all of its pre-places have a token), which removes one token from each of the transition’s pre-places and adds one to each of its post-places. (No other transition can fire initially.) If it does, in the resulting marking, only the other transition can fire, etc.

Per the Hoare logic presented above, for any places we have the following Hoare triple for the I/O expression , where is a character:

where we use the shorthand for . In the remainder, we will abbreviate the action to and the transition to .

Assuming the Petri net above, the following Hoare triple expresses that a function satisfies the I/O specification denoted by the Petri net:444We use notation to abbreviate function application .

(1)

However, when specifying functions, it is preferable that the Hoare triple itself express any necessary assumptions about the Petri net, like so:

This specification universally quantifies over the set of places , the set of transitions , and the places , , and . It is easy to see that this specification is indeed equivalent to specification 1 above, in terms of the I/O traces is allowed to produce. In the remainder, we will always implicitly universally quantify over the set of places, the set of transitions, and any free metavariables (including ones ranging over places) of a specification.

Consider now the following implementation of function :555We use notation to mean , where does not appear in . Similarly, we use to mean and to mean .

We can verify that this function satisfies its specification using the Hoare rules from Figure 2. Such a proof is commonly summarized as a Hoare proof outline that mentions the most salient intermediate assertions, like so:

3.3.1. Underspecification

One can easily express specifications that allow multiple behaviors, by specifying a Petri net where there are multiple paths between the start and destination places. For example:

The corresponding Hoare triple, along with one example implementation that satisfies it, is as follows:

3.3.2. Compositionality

The approach allows one to define composite I/O actions on top of primitive ones, such that client code need not be aware of whether a given action is primitive or not. For example, we can define a composite I/O action as follows:666We use notation to abbreviate the corresponding combinations of , , and .

Notice that if , then . Notice also that verification of client code can proceed as if were a primitive I/O tag instead of a function, and were a primitive I/O expression instead of a function application, and referred directly to a transition of the Petri net instead of being a predicate defined on top of it.

3.3.3. Input

The approach allows one to express input-dependent output requirements, as well as assumptions about the input that will be received. For example, assume . Then for all characters we have the following Hoare triple:777Notation abbreviates transition .

The following specification expresses that function shall perform a action, that this action’s result shall be a lowercase letter (a constraint on the environment), and that the program shall subsequently output the uppercase version of that letter:

(For clarity, we here show the universal quantifications that we will usually leave implicit.) Any particular result-deterministic Petri net that satisfies the precondition has only one transition starting in . However, since the specification is universally quantified over all such Petri nets, it implies that the program properly handles all 26 letters.888The restriction to result-deterministic Petri nets is implied by the semantics of Hoare triples given in §3.2.

3.3.4. Specification-level concurrency

Suppose we want the program to read two characters and print them back to us. We do not want to force the program to print the first character before it reads the second character. We even want to allow the program to read the second character while, concurrently, it is printing the first character.999We treat concurrency in the program formally in §7. A Petri net that expresses this specification is as follows:

split

join

When the transition fires, it consumes the token at and produces two tokens: one at and another one at . (Notice that this specification even allows the program to print the first character (and the second character!) while reading the first character, but of course, that is not physically possible.)

The corresponding separation logic specification, with a matching proof outline, is as follows:

A somewhat more realistic version of is one that reads characters forever and prints them back at its leasure:

where

Here, we intend the weakest solution of these equations; they describe an infinite Petri net. denotes an infinite sequence of characters.

4. Assertions as places

4.1. Motivating example: buffered output

On Unix-like systems, is a C run-time library function that is implemented in terms of the system call, which writes a sequence of characters:

Note: we assume that the effect of is indistinguisable from that of ; to model this, we take as a primitive I/O tag and we define in terms of it. Accordingly, in the formal setting we assume is some function implemented in terms of primitive I/O expressions.

Since system calls are expensive, buffers output in a global variable:

The specification for that we show here does not hide the details of how it is implemented. For example, it names the global variable . We would like to define predicate such that this implementation satisfies the simple, abstract specification for that we showed on p. 3.3. Notice that this requires that we unify the postcondition above with , for some place . Clearly, in the approach of §3, that is impossible, since constrains only the marking, not the heap.

4.2. Assertions as places

What we want is a specification for that looks exactly like the one on p. 3.3, and that therefore allows the client program specifications and proofs shown in §3.3, but which at the same time can be unified with the specification for the buffering implementation shown above.

The main contribution of this paper is the observation that we can achieve this by introducing, to complement the existing primitive notion of places and the existing primitive assertion form, an abstract notion of places, and an abstract version of the assertion form, where places are assertions, and simply means .

We can then unify the abstract reading of the specification of on p. 3.3 with the one above by defining as follows (where ):