I Introduction
Ia What is policyrich routing?
This paper proves new, general results about the convergence of asynchronous routing protocols in the Distributed BellmanFord (DBF) family, which includes distancevector (RIPlike) and pathvector (BGPlike) protocols. In particular, the results apply to what we call policyrich protocols, and so we begin by informally explaining this terminology.
Suppose that a router participating in a DBF computation has a best route to a destination . If it then receives a route for from an immediate neighbour it will first apply some policy associated with that neighbour and produce a candidate route . It will then compare with to determine which is best. Let us denote the outcome of this selection process as . A very simple example is the shortestpaths problem where is just an integer representing distance, for some weight , and .
If we dig deeply into the classical theory underlying bestpath algorithms — both synchronous [1, 2], and asynchronous [3] — we find that it always assumes the following equation, or something equivalent, must hold:
(1) 
This property is referred to as distributivity. The lefthand side of the equation can be interpreted as a decision made by a router sending routes and while the righthand side is a decision made by the neighbour receiving those routes. Assuming the equality holds, the classical theory proves that routing protocols arrive at globally optimal routes — where every pair of routers use the best path between them.
By a policyrich language we mean one in which distributivity does not hold. A clear example of how distributivity violations might arise in routing can be seen in the use of route maps which are functions (scripts) that take routes as input and return routes as output. For example, if and are route maps, then we might define another route map as:
(2) 
where is a predicate on routes (such as “does this route contain the BGP community 17?”). To see how easily distributivity can be violated, suppose that:
Then the lefthand side of Equation 1 is:
while the righthand side becomes:
For Eq 1 to hold we need , which may not be the case (indeed, if were always true, then there would be no point in defining !). Perhaps the most common example of such conditional policies is route filtering, where is equal to the invalid route.
A specialist schooled in the classical theory of path finding might be tempted to forbid the use of such “broken" policies when using the BellmanFord or Dijkstra algorithms. Yet, once again, practice has outstripped theory. The Border Gateway Protocol (BGP) [4] — a key component of the Internet’s infrastructure — is a policyrich routing protocol.
We argue that today’s BGP is in fact broken since it is possible to write policies that can result in anomalous behaviour such as nonconvergence [5, 6] and multiple stable states [7]. The latter are problematic as the extra stable states are nearly always unintended and often violate the intent of policy writers. Leaving an unintended stable state requires manual intervention and, in the worst case, a high degree of coordination between competing networks. This phenomenon is colloquially referred to as a BGP wedgie.
Hence a pertinent question is how to tame the policy language of BGPlike protocols to ensure good behaviour? Of course we could mandate that all protocols must conform to Equation 1. However, we would then not be able to implement typical interdomain policies that are based on commercial relationships [8, 9]. In addition, something as simple as shortestpath routing with route filtering would be banned.
Gao & Rexford [10] showed that the simple commercial relationships described in [8, 9], if universally followed, are enough to guarantee convergence. However their model is BGPspecific and gives us no guidance on how policyrich protocols should be constrained in general. Furthermore their conditions impose constraints on the topology of the network and therefore they require continuous verification as the topology changes.
Instead an alternative middle ground for generic policyrich protocols has been achieved for both Dijkstra’s algorithm [11] and the DBF family [12]. Rather than insisting on distributivity, we require that for all routes and policies we have:
(3) 
In other words, applying policy to a route cannot produce a route that is more preferred. Although Equation 3 is sufficient for Dijkstra’s Algorithm [11], it must be strengthened for DBF algorithms [12] to:
(4) 
That is, applying policy to a route cannot produce a route that is more preferred and it cannot leave a route unchanged. We call policy languages that obey Equation 3 increasing and those that obey Equation 4 strictly increasing. Note that if policies and are strictly increasing, then the conditional policy defined in Equation 2 is also strictly increasing. In other words, a strictly increasing policy language remains strictly increasing when route maps are introduced.
However, without distributivity we can no longer achieve globally optimal routes and so we must be content with locally optimal routes [11] – each router obtains the best routes possible given the best routes revealed by its neighbours.
It is natural to ask if the strictly increasing condition is too strong and prohibits desirable policies? Sobrinho [12] shows that it is the weakest possible property that guarantees a pathvector protocol always converges. This implies that if the policy language is not strictly increasing than there exists a network in which the protocol diverges.
Nonetheless, as we discuss in Section V, the policy language of BGP is not strictly increasing, and there is no known way of making it so without removing the ability of AS’s to hide commercially sensitive information from each other. Therefore, the next natural question to ask is: given a nonstrictly increasing policy language, which networks does the protocol converge over? Sobrinho shows that a network being free with respect to the policy language is a sufficient and necessary condition for a DBF protocol to converge over that network.
IB Our contributions
IB1 Answers to bigpicture questions
Given these prior results, the reader might be tempted to assume that the convergence of standard DBF routing protocols is a solved problem. However, this is far from the case. In this paper, we address the following open questions:

[label=Q0)]

Sobrinho’s proofs only guarantee that the protocols converge. Do they always converge to the same stable state?
Answer: We prove that the answer is yes. From a theoretical perspective, this implies that the set of DBF protocols that converge and the set of DBF protocols that converge deterministically are the same. From a practical perspective, this implies that as long as a DBF protocol is guaranteed to converge then you don’t have to worry about problems like BGP wedgies.

The proofs of Sobrinho and GaoRexford both assume TCPlike inorder, reliable communication between routers. Is this a fundamental requirement for convergence?
Answer: We prove that the answer is no, by showing that the results still hold for a weaker, UDPlike model of communication in which route advertisements may be delayed, lost, reordered and duplicated. From a theoretical perspective, this provides strong evidence that the difficulty of solving a routing problem in a distributed environment is entirely independent of the communication model used. This is not the case in other distributed protocols, e.g. consensus protocols [17].

Sobrinho reasons about pathvector protocols. Do there exist corresponding results for distancevector protocols?
Answer: we show that they do. In particular, that distancevector protocols converge over strictly increasing algebras/free networks if the number of possible pathweights assigned to routes is finite. From a practical perspective this implies the feasibility of policyrich distancevector protocols, a relatively underexplored design space.
IB2 Improvements to the model
In addition to the new highlevel results above, we also make several important contributions to the underlying mathematical model for DBF protocols. A more technical discussion of these contributions can be found in Section IIF.

[label=M0)]

We propose a new algebraic structure for modelling policy languages which is both simpler and moreexpressive than that of Sobrinho. In particular, it contains only 5 algebraic primitives instead of 8, and is capable of naturally modelling pathdependent policies such as route filtering and pathinflation.

We identify that a DBF protocols are a single instance of a much broader class of asynchronous iterative algorithms, for which there exist prior models and convergence results. From a theoretical point of view, this allows us to cleanly abstract the underlying routing problem being solved from the distributed environment in which it is being solved. Practically, it greatly simplifies the proofs and allows us to prove asynchronous convergence purely by reasoning about the behaviour of the protocol in an entirely synchronous environment. In contrast, previous work [10, 12] has typically created their own custom model of message passing in an asynchronous environment and therefore have to directly reasoning about message timings and have avoided reasoning about unreliable communication.

Thanks to this existing theory, we construct a mathematical model of an abstract DBF routing protocol that is the first to precisely describe the entire evolution of the protocol, including exact message timings, routers and links failing, routers and links being added, and routers being reset. As far as we aware, this is therefore the first fully provably correct executable model of an abstract DBF protocol. In contrast, previous proofs have focused on proving results about a snapshot of a single network configuration in which all routers, links and policies remain static.
IB3 Formal verification
In the past decade there has been a strong trend towards the verification of infrastructurerelated software in areas as diverse as operating systems [18], compilers [19, 20] and networking [21, 22].
In line with this trend we have fully formalised all the results in this paper in the Agda theorem prover [23]. As Agda is capable of expressing both proofs and programs, and as our model is executable, this constitutes the first ever fully formalised, executable model of a general DBF protocol as far as we are aware. The advantages of this formalisation effort are as follows:

[label=V0)]

It provides a much stronger guarantee of the correctness of our proofs. Reasoning about algorithms in distributed environments is notoriously tricky, and as Sobrinho himself says in [12], his proof of convergence is only “semiformal”. Having a computer check our proofs is a valuable and powerful assurance as to their correctness.

The generality of our verified proofs means that they also function as a library that can be used by others to rapidly prototype, explore the behaviour of and formally verify the correctness of novel policy languages. This is a significant step towards reusable and modular formal verification of routing protocols. The library is freely available [24], and we hope that it will prove useful to the community.

To demonstrate the utility of our library, we construct a model of a pathvector routing protocol that contains many of the features of BGP including local preferences, conditional policy, path inflation and route filtering. We formally verify in Agda that the policy language is strictly increasing, and therefore no matter what policies are used the protocol will always converge deterministically to a single stable state given a sufficient period of stability. Such a protocol is therefore provably safebydesign.
IC Road map
In Section II, we use an algebraic approach inspired by that of Sobrinho [12] and existing literature on asynchronous iterative algorithms [25], to construct an executable model of an abstract, fully asynchronous, distributed DBF routing protocol. This approach provides a clear, implementationindependent specification of the family of protocols which we can reason about. In Section III we discuss how our novel use of an existing theory of asynchronous iterative algorithms does much of the heavy lifting, and finish by applying using existing results from this field to prove our new results about distancevector and pathvector protocols. In Section IV we describe our Agda formalisation of these results, and in Section V we construct a safebydesign algebra using the library.
ID Previous versions
Ii Model
In this section we construct a model of an arbitrary DBF routing protocol. For the reader’s convenience, a glossary of all the notation used in this and later sections can be found in Appendix A.
Iia Generality vs implementation details
In any mathematical model of a realworld system, there is a fundamental tension between capturing the most general form of the system and capturing every detail of the implementation. More implementation details can always be added to better model a particular system, at the cost of losing the ability to apply the work more generally.
In this paper we err on the side of generality. While we acknowledge that BGP is the dominant policyrich pathvector protocol, the aim of our work is not only to inform how BGP policies may be constrained to provide better guarantees, but also to guide the design of future policyrich protocols.
For example we do not explicitly model the hardstate nature of BGP. Instead we use a very general model of asynchronous computation which we believe can easily capture a hardstate protocol (this intuition is not fully formalised in this work). Had we explicitly modelled a hardstate protocol, it would prevent us from applying our work to softstate protocols as well.
IiB Paths
Prior to anything else, it is important to discuss exactly what is meant by a path in the context of a distributed, asynchronous routing protocol. Traditionally paths are defined with respect to some graph . However because a network’s topology changes over time, the path along which an entry in a router’s routing table was generated may not correspond to a valid path in the current topology. Furthermore, and perhaps counterintuitively, such a path may be formed from an amalgamation of paths from previous topologies, and therefore may not even have existed in any single past topology. Consequently, out of necessity, our notion of a path cannot be tied to the network’s topology.
Therefore, we begin by defining as the set of all valid router identifiers. An edge is an arbitrary pair of router identifiers, and therefore the set of edges is . A path is a defined to be a sequence of contiguous edges. A cycle is a path which finishes at the same router it started at. A simple path is a path that doesn’t contain any cycles. For our purposes, we also consider an additional simple path , which represents the absence of a path. We refer to the set of simple paths as .
IiC Routing algebras
In order to model all possible DBF protocols rather than one particular protocol, we first abstract over the routing problem being solved by representing it as a routing algebra.
Definition 1 (Routing algebra).
A routing algebra is a tuple where:

is the set of pathweights. A pathweight is the data that the routing protocol associates with each candidate path. For example in a shortestpath protocol the pathweights would be the lengths of the paths, e.g. .

is the choice operator, which given two pathweights returns the preferred pathweight. In the introduction, this was informally referred to as “best”. In a shortestpath protocol this would be the operator.

is a map from an edge to the set of policy functions that may be used on that edge. Each policy is a function , which when given a pathweight returns a new pathweight formed by extending the original path by the edge . For notational convenience we define the complete set of policies as .

is the pathweight of the trivial path from any router to itself. In a shortestpath protocol this is 0.

is the invalid pathweight, representing the absence of a path.
which we assume to always have the following minimal properties:

[label=R0]

is associative  the order doesn’t matter when making a sequence of choices between pathweights.

is commutative  the order doesn’t matter when choosing between two pathweights.

is selective  choosing between two pathweights always returns one of the two pathweights.

is an annihilator for  the trivial pathweight is more desirable than any other pathweight.

is an identity for  any pathweight is preferable to the invalid pathweight.

is a fixed point for all policy functions  the extension of the invalid pathweight is also the invalid pathweight.
Table I contains examples of simple routing algebras representing a variety of common routing problems. A much more complex routing algebra that is based off of BGP is discussed in detail in Section V.
Use  

shortest paths  
longest paths  
widest paths  
most reliable paths 
e.g. .
Pathvector protocol are a subclass of DBF protocols which track the path along which a pathweight was generated. Routers then use this information to eliminate any pathweights that were generated via a cycle. It is well known that pathvector protocols have better convergence behaviour. To allow us to reason about such protocols we now define a subclass of routing algebras that can be used to model pathvectorspecific behaviour.
Definition 2 (Path algebra).
A path algebra is a routing algebra equipped with an additional function that obeys the following properties:

[label=P0]

A pathweight has the invalid path iff it is the invalid pathweight.

The path of the trivial pathweight is the empty path.

Applying an edge’s policy function to a pathweight simply extends the pathweight’s path by that edge, unless it does not result in a nonsimple path in which case it becomes the invalid path.
The function allows us to extract the path along which the pathweight was generated without revealing how the path is stored internally inside the pathweight. The advantages of this abstraction are discussed at the end of this section. Assumptions P1 – P3 ensure that the algebra’s implementation is tracking the paths correctly. In P3 the first case ensures that pathweights generated along paths with cycles in them are eliminated. The second case ensures that invalid routing announcements are also eliminated.
Note that assumption P3 is the primary reason that we require that each edge comes with its own set of policy functions in Definition 1. If instead we had only required the existence of a single set of policy functions (i.e. ) then we would be unable to express P3.
IiD Synchronous DBF protocol
Now that we can model what problem our abstract DBF protocol is solving, we move on to modelling how it solves it. We start by defining a synchronous version of the protocol where update messages between routers are exchanged instantaneously and in parallel. Let be the finite set of routers that will ever participate in the protocol and let . We represent an instantaneous snapshot of the network topology by an adjacency matrix where is the current policy function on the link from router to router . Note that in practice the policy function may be the composition of the export policy of router and the import policy of router . The absence of a link from to is represented by the alwaysinvalid policy which is defined as .
The global state of the synchronous protocol can be entirely represented by the product of the state of each router’s routing table. We therefore represent an individual state by a matrix where row is router ’s routing table and so the element is the pathweight of ’s best current route to router . We will use to denote the set of routing table states (i.e. the set of vectors over of length ), and to denote the set of global states (i.e. the set of matrices over ).
We define the sum of two matrices and as:
and the application of to as:  
In one synchronous round of a distributed BellmanFord computation every router in the network propagates its routing table its neighbours who then update their own tables accordingly. We model this operation as :
where is the identity matrix:
The nature of the underlying computation becomes clearer when expanding out the definition of a single element of :
(5) 
Router ’s new route to is the best choice out of the extensions of the routes offered to it by each of its neighbours .
We can now model the evolution of the synchronous version of protocol, , as repeated applications of .
Definition 3 (Synchronous state function).
Given a network topology and an initial state , the state of the synchronous DBF protocol at time is defined as:
IiE Asynchronous DBF protocol
Although we have defined a synchronous model above, in reality DBF protocols are distributed and asynchronous. Update messages are dispatched at nondeterministic intervals and may be delayed, reordered, duplicated or even lost along the way. New routers and links may be added to the network, existing routers and links may fail, and routers may be rebooted and reset to their original state. Reasoning about the outcomes of such unpredictable events is known to be extremely challenging.
One advantage of first defining the synchronous model is that it highlights that the underlying algorithm is inherently iterative. Consequently, it is easy to recognise that a DBF protocol is simply a special instance of the family of dynamic asynchronous iterative algorithms. We can therefore use existing work [28, 25] that constructs mathematical model of such iterations. We now describe a specialisation of the model presented in [25] to DBF protocols.
The model assumes a discrete and linear notion of time,
, denoting the times of events of interest in the network (a routing table updating, a router failing etc.). Layered on top of time is the notion of epochs,
, which represent contiguous periods of time during which the network topology remains unchanged, i.e. no links or routers fail or are added and no policies are changed. Given this, we can now model a dynamic network, , as a function from epochs to adjacency matrices where is the topology of the network during epoch .The nondeterministic evolution of the protocol is described by a schedule.
Definition 4 (Schedule).
(Definition 9 in [25]) A schedule is a tuple of functions where:

is the activation function, where is the set of routers which update their routing table at time .

is the data flow function, where is the time at which the route advertisements used by router at time was sent by router .

is the epoch function, where is the epoch at time .

is the participants function, where is the subset of routers currently participating in the protocol during epoch e.
such that:

[label=S0]

information only travels forward in time

epochs increase monotonically
This is a very weak model of asynchronous communication. As shown in Figure 1, nothing forbids the data flow function from delaying, losing, reordering or duplicating messages. It also distinguishes between a router being a member of the network but not currently participating in the protocol. This allows us to model messages that continue to arrive from the previous epoch, e.g. router can receive outdated route advertisements from router even after router has ceased to participate.
However, it does mean that we need to distinguish between the topology of network and the topology as seen by the active participants. In particular we need to enforce that information only flows between two participating routers.
Definition 5 (Participating topology).
Given a network , an epoch and a set of participants , we define to be the participating topology:
Given a schedule, for notational convenience we also define , the set of participants at time , and , the current participating topology at time , as:
Using this machinery it is now possible to define the full asynchronous state function, , in terms of the original iteration function .
Definition 6 (Asynchronous state function).
(Definition 10 from [25] with , ) Given a dynamic network , an initial state , and a schedule , the state of the asynchronous DBF protocol at time is defined as:
where .
This models the evolution of the protocol as follows. If a router is not currently participating then it adopts its nonparticipating state . If a router has just begun to participate, either because or because it was not participating at the previous time step, it adopts the initial state . If the router is a continuing participant and is currently inactive then its state remains unchanged from that at the previous time step, . Otherwise, if it is currently active, it updates its state in accordance with router ’s local view, , of the global state as currently advertised by its neighbours. More concretely, is the last route advertisement for router that router received from its neighbour .
Note that this is a straightforward generalisation of the synchronous model as we can immediately recover from by setting , , and i.e. there is a single epoch in which at each time step, every router is both participating and active and all messages only take a single time step to propagate.
IiF Novel contributions
We now recap the main advantages of this new model over models proposed in previous work.
Simpler, more general algebraic structure
The algebraic structure we propose in Definition 1 is both more general and simpler than that proposed by Sobrinho [12], which in turn was inspired by the classical semiring approach [1, 29]. The main difference is in how we model pathweight extension. In semiring algebras it is modelled by a second binary operator . However this is incapable of modelling modern routing policies such as those described in Sections I & V. Sobrinho generalised this binary operator by introducing labels , signatures , an operator and a function mapping signatures to pathweights. However, by modelling extension policies as a set of functions, , and as demonstrated in Section V, we retain the ability to model modern routing polices while reducing the size of the algebra from 8 primitives to 5 primitives.
Paths as part of the algebra
The second way in which the algebra is more general is that, by assigning each edge its own set of extension functions, we model the tracking of paths in pathvector protocols inside of the algebra itself. In contrast, Sobrinho [12] models the paths as part of the algorithm, i.e. in the equivalent definition of . This means that the paths along which the pathweights were generated are not provided as inputs to his and operators. Therefore they cannot alter or make decisions based on those paths. Consequently, unlike ours, Sobrinho’s approach cannot model policies such as route filtering or path inflation in BGP.
Abstraction of paths
A third advantage of our new Definition 2 of a path algebra is that it only requires that the function can extract a simple path from a pathweight, but does not force the protocol to reveal how paths are actually stored internally. This abstraction results in proofs that are far more robust to variations in how paths are stored internally by the protocol. For example, in Section V we show that our algebras can model BGPlike path inflation by simply adjusting the function to strip out the inflated ASs, without having to adjust the proofs of convergence. In contrast, the model of pathvector protocols in Sobrinho [12] uses a concrete definition of paths, and therefore is incapable of modelling path inflation without altering all the proof of convergence.
Use of existing asynchronous theory
The additional complexity introduced by the more expressive dynamic asynchronous model , such as routers failing and rebooting, at first glance appears daunting. However, as we will discuss further in Section III, our use of the existing theory of dynamic iterative algorithms means that we have access to a range of powerful theorems that, impressively, are capable of totally abstracting away the asynchronicity of the system.
Executable model of the protocol
Sobrinho [12] only provides an abstract temporal model of individual events that occur during the operation of the protocol. In contrast, as far as we are aware, ours is the first fully formal and executable model of the full dynamic behaviour of an abstract DBF protocols in which routers and links can fail, join and reboot. This means that our model can be run and its behaviour observed, and that our proofs can be written in a style amenable to the formal verification discussed in Section IV.
Iii Convergence results
Iiia Some useful concepts
Initially, we define some concepts that, although not required for constructing the model in Section II, are used in stating and reasoning about its convergence behaviour.
IiiA1 Ordering over path weights
Given any routing algebra , the choice operator induces a preference order over pathweights as follows:
Definition 7 (Preference relation over pathweights ).
A pathweight is preferred to pathweight , written , if is chosen over :
As is associative, commutative and selective, we have that is a total order and that for all pathweights we have .
IiiA2 Assignments
In our model which router is using which pathweight is implicitly defined by the pathweight’s location in the routing state (i.e. given a routing state then the pathweight is being used by router ). Nonetheless a more explicit notion of which router is using which pathweight is crucial for stating sufficient and necessary conditions for the convergence of the model over a given network. We shall refer to such a concept as an assignment.
Definition 8 (Assignment).
An assignment is a pair of a router and a pathweight . The set of assignments .
Note that an assignment is a generalisation of the notion of a couplet defined by Sobrinho [12]. Whereas a couplet stores the entire path along which the pathweight was generated, an assignment only stores the first router in the path. This generalisation is necessary as we seek to reason about distancevector as well as pathvector protocols, and in the former case we do not have access to the full path.
Several binary relations may be defined over the set of assignments. The first is the lifting of the preference order over pathweights to assignments.
Definition 9 (Preference relation over assignments).
An assignment is preferred to assignment , written , if they have the same router and the pathweight of is preferred to the pathweight of :
Note that unlike which is a total order over pathweights, is only a partial order over assignments, as it can only relate assignments to the same router. Intuitively, this makes sense as routers in a DBF protocol only compare paths that begin at themselves.
Definition 10 (Extension relation).
An assignment extends assignment in the current topology , written , if the extension of the noninvalid pathweight of across the relevant link is equal to the pathweight of :
In the context of a path vector protocol, the axioms of a path algebra are sufficient to guarantee that if then .
Definition 11 (Threatens relation).
An assignment threatens assignment in the current topology , written , if the extension of the noninvalid pathweight of across the relevant link is preferred to the pathweight of :
Intuitively means that after the next update router may switch away from and instead use a pathweight that is equal to or preferred to . It is easy to see that if then .
IiiA3 Nonparticipating nodes
As seen in Definition 5, the constraint that nonparticipating routers do not advertise routes to or receive routes from other routers is enforced at the level of the adjacency matrix and therefore the iteration function . Consequently, after a single iteration the state of any nonparticipating router’s routing table will always be the corresponding entry of the identity matrix, i.e. the trivial route to itself and invalid routes to all other routers in the network. In [25] such states are said to be accordant with the current set of participants.
Definition 12 (Accordant states).
(Definition 15 from [25] with ) The set of routing states that are accordant with set of participants , , is defined as:
IiiB Definition of convergence
Before proving convergence results about our model, we must first establish what does convergence even mean for a dynamic, asynchronous, iterative algorithm such as a DBF protocol? In this section, will build up a suitable definition by discussing in turn the complications introduced by each of these adjectives.
Iterative
The definition of convergence for a synchronous iterative algorithm is relatively simple. After a finite number of iterations, the algorithm must reach a stable state such that further iterations result in no further change. In terms of the synchronous model defined in Section IID, this would translate to:
The resulting stable state is a fixed point for , as:
From the perspective of a routing protocol, being a fixed point is equivalent to the statement that in state no router can improve its selected pathweights by unilaterally choosing to switch. Such a state can therefore be viewed as a locally optimal solution to the optimisation problem of finding the best consistent set of path choices in the current network topology. As will be discussed further in Section IIID, whether such a state is necessarily globally optimal depends on the properties of the routing algebra being used.
Asynchronous
At first glance, it appears relatively easy to adapt the above definition to a basic asynchronous model with only a single epoch: simply require that for every schedule the asynchronous iteration reaches a stable state. Unfortunately due to the asynchronous and distributed nature of the computation, and as discussed in Section IIE, our definition of a schedule does not assume reliable communication between routers. Consequently, in an arbitrary schedule a router may never activate or a pair of routers may never succeed in communicating. In such a schedule, it is clearly impossible for the protocol to converge to the desired stable state. Therefore, we need to restrict ourselves to only requiring that the protocol converges over reasonable schedules.
As long established in the asynchronous iterations literature [28], the right definition of “reasonable" is that the schedule contains pseudocycles. A pseudocycle is a period of time within an epoch such that for every participating router: all messages in flight to that router at the start of the pseudocycle are either dropped or arrive and, after this has occurred, the router performs at least one update. A formal definition of this concept is provided in Definition 13 in [25]. Informally, a pseudocycle can be thought of the asynchronous counterpart of a single synchronous iteration. Lemma 8 in [25] makes this intuition formal by proving that during a pseudocycle, the asynchronous model makes at least as much progress towards the stable state as the synchronous model would with a single synchronous iteration.
Another aspect that we must consider when moving from the synchronous to the asynchronous model is that of nondeterminism. As discussed in Section IA, BGP suffers from nondeterministic convergence where the protocol may end up in multiple stable states, and which stable state is reached depends on the schedule and the initial state, rather than just the topology. This problem is known colloquially as a BGP wedgie [7]. Our definition of convergence should therefore require it to be deterministic, i.e. in each epoch only one stable state may be reached and that it is dependent only on the topology and the set of participants.
Dynamic
Finally, we now consider the full dynamic model with multiple epochs. In this scenario the network topology may change so frequently (i.e. the epochs may be so short), that the protocol never gets sufficient time to converge to a solution. Therefore we would like our definition of convergence to only require that the protocol will converge if given a sufficient period of stability. It is suggested in [25] that dynamic iterations should therefore be referred to as convergent rather than saying that they converge.
The final issue is that the protocol may only be convergent over some, rather than all, epochs and sets of participants. As shown by Sobrinho [12] and discussed further in Section IIID, DBF protocols only converge over network topologies that are free. Therefore the definition of convergent must be granular enough to express that the protocol is only convergent over a subset of epochs and sets of participants. A pair containing an epoch and a set of participants is referred to as a configuration. The set of all configurations is therefore defined as:
Taking all the above points into account, we therefore arrive at the following definition of what it means for our protocol to be convergent:
Definition 13 (Convergent).
(Definition 14 from [25]) The DBF protocol is convergent over a set of configurations iff:

for every configuration there exists a fixed point for and a number of iterations .

for every initial state , schedule and time then if and the time period contains pseudocycles then for every time such that and then
The essence of this definition is that for every valid configuration there exists a fixed point and a number of pseudocycles such that if, at an arbitrary point in time the configuration of the current epoch is in the set of valid configurations and the schedule contains that number of pseudocycles, then after those pseudocycles the protocol will have reach the fixed point and will remain in that fixed point until the end of the current epoch.
IiiC An existing convergence theorem
One of the most important advantages to using an established model of asynchronous iterations in Section IIE is that there already exist theorems describing sufficient conditions of such an iteration being convergent. The most powerful (and perhaps initially surprising) feature of these results is that the sufficient conditions only involve properties of the functions being iterated, , rather than the full asynchronous iteration, . Consequently one can use them to prove that is convergent without ever having to directly reason about the unreliable, asynchronous and dynamic nature of the underlying network, e.g. messages being lost or reordered, routers and links failing or being added.
In this paper, we will use the AMCO conditions from [25]. For a broader survey of the range of alternative sufficient conditions available see [28].
Definition 14 (AMCO, Definition 23 in [25]).
The functions are a set of asychronously metrically contracting operators (AMCO) over a set of configurations if for every epoch and set of participants such that then for all there exists a distance function such that:

[label=D0]

is indiscernable

is bounded
and if then:

[resume, label=D0]

is strictly contracting on orbits over

is strictly contracting on fixed points over :

enforces accordancy:
The properties required of an AMCO may seem complicated at first glance but each have clear interpretations. Firstly for every configuration in there must exist a function which measures the distance between two routing table states (although it need not satisfy the formal definition of a metric). D1 requires that the distance between two states is zero iff the states are equal, and D2 that there exists a maximum distance. Distances over routing tables, , are then naturally lifted to distances over routing states by calculating the maximum distance between the participating routing tables. D3 requires that the distance between the states after consecutive iterations strictly decreases, and D4 that if there exists a stable state then the distance to it strictly decreases after each iteration. Finally, D5 requires that applying the operator to an accordant state must always result in another accordant state.
Theorem 1.
If is a dynamic AMCO over a set of configurations then is convergent over .
Informally a synchronous version of the proof runs as follows: D3 require that each synchronous iteration moves the current state an increasingly smaller distance. As the distances are all bounded natural numbers by D2 then the distance must eventually reach zero, which by D1 means that one has reached a stable state. D4 then guarantees that this stable state is unique. With some nontrivial work, it is possible to lift this synchronous argument to an asynchronous argument because the distance function over routing states is decomposable into individual distance functions over routing tables.
The key point to take away is that by using this theorem in our later proofs, we reduce the problem of proving our model of the DBF protocol, , is convergent to merely constructing a suitable distance function which interacts with in the right way.
IiiD Types of routing algebras
In this section, we adapt the hierarchy of necessary and sufficient conditions for convergence established by Sobrinho [12] to our new simpler algebraic model^{1}^{1}1Our terminology differs slightly from that of Sobrinho as we use the traditional algebraic/order theoretic names for these conditions. In particular we use distributive instead of isotonic and increasing instead of monotonic. This is because the order theoretic notion of monotonicity, , is much more closely related to the notion of distributivity, , than increasingness, .. The key insight of Sobrinho is that each of the routing algebra axioms in Definition 1 only reference either choice or extension, and that is the relationship between them that determines the convergence behaviour of the protocol.
As discussed in Section IA, the goldstandard are distributive algebras:
Definition 15 (Distributive algebra).
A routing algebra is distributive if:
As was proven many times in the classic algebraic routing literature and, again by Sobrinho, then if the algebra is distributive then any stable state will necessarily be a globally optimal state in which every pair of routers is assigned the optimal path between them.
The next most desirable property for the algebra to have is to be increasing/strictly increasing:
Definition 16 (Increasing algebra).
A routing algebra is increasing if:
Definition 17 (Strictly increasing algebra).
A routing algebra is strictly increasing if:
Sobrinho proved that the algebra being strictly increasing is a necessary and sufficient condition for a pathvector protocol to converge to some stable state in a given epoch. However, if the algebra is not distributive then that stable state will only be locally optimal rather than globally optimal, and some pairs of routers will end up using suboptimal paths.
Finally there is the freeness property over a particular network topology:
Definition 18 (Free network topology).
A network topology is free with respect to the routing algebra if there does not exist a cycle and a set of pathweights such that:
where is calculated mod .
Intuitively a network is free if there does not exist a cycle where each router’s current assignment can threaten the assignment of the previous router in the cycle. Sobrinho proved that the network topology being free with respect to the underlying algebra is a necessary and sufficient condition for a pathvector protocol to converge over that network, and that in the shortestpaths algebra this is equivalent to there being no negative weight cycles. Let be the set of configurations in which the participating topology is free, i.e.
We now prove a couple of small lemmas showing some of the relationships between these properties.
Lemma 1.
If a routing algebra is strictly increasing then every network topology is free.
Lemma 2.
If a path algebra is increasing then it is also strictly increasing.
Proof.
Consider arbitrary and and such that . We already have that as the algebra is increasing so it remains to show that .
IiiE DBF convergence results
We now proceed to state our main convergence results, the proofs of which may be found in the appendices. We start by considering distancevector protocols.
Theorem 2.
If is a routing algebra and is finite then is convergent over .
Proof.
See Appendix B. ∎
Theorem 2 says that for any distancevector protocol with a finite set of pathweights then for any epoch and set of participating routers such that the network topology is free and given a sufficient period of stability in which the participating routers continue to activate and communicate, the protocol will always converge to the same stable state no matter the exact timing of messages and even in the presence of unreliable communication.
A more useful result for strictly increasing, finite algebras is then immediately obtainable which shows that such a distancevector protocol is always convergent.
Theorem 3.
If is a strictly increasing routing algebra and is finite then is always convergent.
Proof.
In practice the finiteness condition is restrictive and excludes many routing algebras of interest. For example even the shortestpath algebra uses the infinite set as path weights. However recall that Theorems 2 & 3 guarantee that, given a sufficient period of stability, the protocol will reconverge even in the presence of messages from previous epochs. On the other hand shortestpath distancevector protocols experience counttoinfinity problems when the state at the start of the epoch contains pathweights generated along paths that do not exist in the current topology. Finiteness is a sufficient condition to reduce counttoinfinity problems to merely counttoconvergence in distancevector protocols. In the real world this is reflected in the design of RIP which artificially imposes a maximum hop count to ensure that is finite.
A common approach to avoid the finiteness requirement (and counttoconvergence issues) is that of pathvector protocols which track the paths along which the pathweights are generated. Pathweights are then removed if their path contains a cycle.
Theorem 4.
Given a path algebra then is convergent over .
Proof.
See Appendix C. ∎
Again a more useful version of this proof for strictly increasing path algebras is immediately obtainable.
IiiF Novel contributions
We now recap the main contributions of these proofs over that of previous work.
Deterministic convergence
Sobrinho’s proofs only guarantee that the protocol will converge, and nothing in his proof forbids the stable state reached depending on the state at the start of the epoch or the ordering of messages between routers during the epoch. In contrast our proofs guarantee that the protocol always converges to the same stable state, no matter the initial state or what the ordering of messages is. This therefore eliminates the possibility of phenomena such as BGP wedgies.
Unreliable communication
The definition of the dataflow function, , in Definition 4, Section IIE means that we only assume unreliable communication between routers. This is in contrast to previous proofs of convergence, for both general DBF protocols [12] and BGP in particular [10], that assume reliable communication. This means that our model and resulting proofs in later sections in theory apply to protocols built ontop of unreliable transport protocols such as UDP [30].
Extension to distance vector protocols
Iv Formalisation in Agda
We have formalised every mathematical result in this paper, down to the most trivial lemma, in the Agda theorem prover. This includes the results about asynchronous iterative algorithms from [25]. As our proofs have been checked by a computer, we are far more confident in their correctness than usual. Even where we omit details or use standard informal mathematical reasoning to improve readability (e.g. Lemma 13 in Appendix C), the proofs are backed by mathematical arguments which are guaranteed to be fully rigorous.
Furthermore the act of formalisation itself was invaluable in creating and shaping these proofs. For example the unstructured way in which we initially laid out the penandpaper proof of Lemma 12 in Appendix C led us to overlook Case 2.3. Only when formalising the result, did we notice that this case remained unproven. This in turn led us to presenting the proof in the much cleaner structure displayed in this paper.
The library of proofs is freely available [24] and is highly modular. To prototype a new policy language users need only define it as an algebra and prove that the algebra obeys the strictly increasing conditions in order to guarantee the resulting protocol is always convergent. Furthermore the protocol is then executable so that its dynamic behaviour can be observed for a given network and schedule. In Section V we give an outline of how to define such an algebra in Agda. We hope that the library’s extensible nature means that it will be of use to the community when designing new policy languages.
V A safebydesign algebra
We now present an example of how one could use our Agda library to develop a safebydesign routing protocol. In particular, we explore the construction of a pathvector algebra that contains many of the features of BGP such as local preferences, community values [31] and conditional policies. These policies can perform operations such as pathfiltering, pathinflation, modifying local preferences and communities. The conditions themselves are implemented using a simple language of predicates that includes the ability to inspect communities. The algebra is a generalisation of the Stratified Shortest Paths algebra [32].
It differs from the algebra underlying today’s BGP in two crucial ways. Firstly, the implementation of the MED attribute violates the assumption that is associative [33]. Secondly, BGP allows ASs to hide their local preferences and set them to arbitrary values upon importing routes from other ASs. This violates the assumption that the algebra is increasing.
Our algebra avoids these two issues by a) ignoring MED and b) having policies that only allow local preference to increase. For convenience, and to mirror the increasing terminology,
Note that we present our algebra only to give a practical example of how our theory can be used, and to show that “most" of the features of BGP are inherently safe. We are not presenting it as a practical solution to these two problems in realworld BGP. We discuss the open question of whether hidden information is compatible with increasing algebras in Section VIB.
Pathweights in our protocol are defined as follows:
[0]dataPathWeight:Setwhere
[0][@l@0][2]invalid[10]:PathWeight
[2]valid[10]:LocalPref→Communities→Path→PathWeight
i.e. there exists an invalid pathweight, and all other pathweights have a local preference, a set of communities and a path. The trivial pathweight, , is defined as:
[0]0#:PathWeight
[0]0#=valid()∅[]
where is the highest possible local preference, is the empty set of communities and is the empty path. The invalid pathweight, , is defined as:
[0]∞#:PathWeight
[0]∞#=invalid
The choice operator, , is defined as:
[0]_⊕_:Op₂PathWeight
[0]x[30]⊕y@(invalid)[50]=x
[0]x@(invalid)[30]⊕y[50]=y
[0]x@(validlcsp)[30]⊕y@(validmdsq)[50]withcomparelm
[0]…tri>l>m[20]=x
[0]…tri<l<m[20]=y
[0]…tri≈l=m[20]withcompare(lengthp)(lengthq)
[0]…[15]tri<p<q[26]=x
[0]…[15]tri>p>q[26]=y
[0]…[15]tri≈p=q[26]withp≤ₗₑₓ?q
[0]…[8]yesp≤q=x
[0]…[8]no[14]q≤p=y
which operates as follows:

If either or is invalid return the other.

otherwise if the local pref of either or is strictly greater than the other return that pathweight.

otherwise if the length of the path of either or is strictly less than the other return that pathweight.

finally break ties by a lexicographic comparison of paths.
We will now construct the set of edge weights . We start by defining a simple yet expressive language for conditions that can be used by our policy language to make decisions.
[0]dataCondition:Setwhere
[0][@l@0][2]_and_[11]:Condition[24]→Condition[37]→Condition
[2]_or_[11]:Condition[24]→Condition[37]→Condition
[2]not[11]:Condition[24]→Condition
[2]inPath[11]:Node[24]→Condition
[2]inComm[11]:Community[24]→Condition
[2]hasPref[11]:LocalPref[24]→Condition
The semantics for these conditionals is defined as follows:
[0]eval:Condition→PathWeight→Bool
[0]eval(pandq)[21]x[36]=evalpx∧evalqx
[0]eval(porq)[21]x[36]=evalpx∨evalqx
[0]eval(notp)[21]x[36]=not(evalpx)
[0]eval(inComm[18]c)[21](validlcsp)[36]=⌊c∈?cs⌋
[0]eval(inPath[18]i)[21](validlcsp)[36]=⌊i∈?p⌋
[0]eval(hasPref[18]v)[21](validlcsp)[36]=⌊v≟l⌋
[0]eval_[21]invalid[36]=false
We next define the policy language as follows:
[0]dataPolicy:Set₁where
[0][@l@0][2]reject[13]:Policy
[2]decrPrefBy[13]:ℕ→Policy
[2]addComm[13]:Community→Policy
[2]delComm[13]:Community→Policy
[2]inflate[13]:ℕ→Policy
[2]_;_[13]:Policy→Policy→Policy
[2]if_then_[13]:Condition→Policy→Policy
The semantics of each type of policy are defined by the function that applies policies to pathweights:
[0]apply:Policy→PathWeight→PathWeight
[0]applyp[21]invalid[37]=invalid
[0]applyreject[21]x[37]=invalid
[0]apply(decrPrefByv)[21](validlcsp)[37]=valid(lv)csp
[0]apply(addCommc)[21](validlcsp)[37]=validl(addccs)p
[0]apply(delCommc)[21](validlcsp)[37]=validl(remove[56]ccs)p
[0]apply(inflaten)[21](validlcsp)[37]=validlcs(inflatepn)
[0]apply(p;q)[21]x[37]=applyq(applypx)
[0]apply(ifcthenp)[21]x[37]=if(evalcx)
[10]
then(applypx)elsex
where the inflate function prepends copies of the path’s source on the front of the path. Note that since the policy language provides no way of increasing a pathweight’s local preference or decrease the length of its path, it is not possible to define a nonincreasing policy.
We define the set of edge weight functions for each edge as:
[0]dataExtension{n}(ij:Noden):Set₁where
[0][@l@0][2]ext:Policy→Eij
and the application of an extension function is defined as:
[0]_▷_:Extensionij→PathWeight→PathWeight
[0](extpol)▷[28]invalid[43]=invalid
[0](extpol)▷[28](validlcsp)[43]with(i,j)⇿?pi∈?p
[0]…no[10]_[16]_[27]=invalid
[0]…yes_[16]yes_[27]=invalid
[0]…yesij⇿p[16]no[22]i∈p[27]=applypol(validlcs((i,j)∷p))
where (i,j)⇿?p tests if the edge is a valid extension of path (i.e. if ), and i∉?p tests whether or not already exists in (i.e. if the resulting path would loop).
The function from pathweights to simple paths required by a path algebra can be defined as:
[0]path:PathWeight→Path
[0]pathinvalid[19]=⊥
[0]path(valid__p)[19]=deflatep
where deflate strips from the path any consecutive duplicate routers that might have been introduced by inflate. The reader might have noted at this point that all our proofs in this paper work over simple paths, yet this algebra internally stores nonsimple paths. However this apparent contradiction is resolved by observing that, as was discussed in Section IIF, the definition of a path algebra only requires that we can extract a simple path from a pathweight, rather than requiring that the pathweight stores a simple path. This demonstrates the power of the path algebra abstraction.
Taken together, the above definitions ensure that the algebra
satisfies all the requirements of an increasing path algebra as defined in Section IIC. For formal proofs of these properties see the Agda formalisation [24].
Our implementation of Theorem 5 in Agda guarantees that a protocol based on this algebra converges from any state to a unique solution even in the presence of message loss, reordering and duplication, i.e. it is impossible to write policies that interfere with convergence.
There are many other features of BGP that are safely increasing but, for space reasons, are not included in this model. For example hierarchical AS paths are possible to add with minor tweaks to the function and the policy language.
Vi Open questions
Via Rate of convergence of pathvector protocols
One aspect of convergence that this paper does not address is how long it takes for convergence to occur. Consider a network of routers. It has long been known that pathvector protocols with distributive, strictly increasing policies require time to converge in the worstcase [1, 2]. However, it was only recently proved that nondistributive, strictly increasing pathvector protocols require time to converge in the worstcase [34]. Meanwhile, a tight bound for the time required for the worstcase convergence of pathvector protocols in free networks is still not known however. Current results have it lying between and .
Another question of interest is that it appears that not all nondistributive, strictly increasing algebras require iterations (e.g. the shortestwidestpaths algebra). We suspect that a careful analysis of policy language features might be able to tease apart distinct classes with respect to worstcase convergence time.
ViB Hidden information
Unlike in (external) BGP, in the algebra described in Section V the local preference attribute is not deleted when exporting a route. This raises a more general issue for routing protocols that allow information to be hidden. It is an open question as to whether it is possible to have increasing algebras with hidden information without requiring global coordination.
Ensuring increasing policies in today’s BGP may require communicating lost information with some other mechanism such as community values. Of course only the relative ranking of local preference values assigned within an AS matter. For example one AS might use a local preference of 100 for its most preferred routes, while another could use 2000. In this context can we ensure increasing policies using only bilateral agreements between neighbouring networks or does it truly require global coordination? If the latter, then a political rather than a technical solution is required.
ViC Verification of datacenter policies
BGP is widely used today to implement (private) connectivity within and between datacenters [35]. In such an environment the network architects have total control of the global topology and therefore hidden information is not an issue. Yet even here we have witnessed the use of conditional policies, combined with filtering and the manipulation of local preference on routes. Perhaps tools such as Propane [36] could be extended to either ensure that all policies are strictly increasing, or at the very least provide warnings when they are not?
Vii Previous versions of this work
A previous version of this work has appeared before in conference proceedings [26]. This version makes the following new contributions, listed in order of appearance:

in this version, the set of policy functions in a routing algebra in Definition 1 are parameterised by edges, which allows us to define a path algebra in Definition 2 without having it depend on , the current topology of the network. Although an apparently minor change, this makes the path algebras significantly easier to work with.

in this version we use the new dynamic model of asynchronous iterations from [25] rather than the static model from [28] which only captures the dynamics of a single epoch. This is not only a more accurate model as it can capture messages from nonparticipating nodes from previous epochs, but it also results in the model being fully executable.
With the exception of Theorems 2 & 4, much of this work appears in Chapters 4 & 5 in the PhD thesis of the first author [27].
Viii Acknowledgements
Matthew L. Daggitt was supported by an EPSRC Doctoral Train ing grant and the ESPRC AISEC grant.
Dr Matthew L. Daggitt received his PhD degree in computer science from the University of Cambridge in 2018 and is currently working as a postdoc at HeriotWatt University, Scotland. His main interests lie in the formal analysis and verification of complex systems, such as networking protocols and machine learning algorithms. 
Timothy G Griffin received his B.S. degree in mathematics from the University of Wisconsin, Madison, in 1979, and M.S. and his Ph.D. degrees in computer science from Cornell University, Ithaca, NY, in 1985 and 1988, respectively. Previous experience includes teaching at UNICAMP in Brazil and more than a dozen years in industry at Bell Laboratories, AT&T Research, and Intel Labs. He joined the Computer Lab at the University of Cambridge in 2005. 
References
 [1] M. Gondran and M. Minoux, Graphs, dioids and semirings: new models and algorithms. Springer Science & Business Media, 2008, vol. 41.
 [2] J. S. Baras and G. Theodorakopoulos, “Path problems in networks,” Synthesis Lectures on Communication Networks, vol. 3, no. 1, pp. 1–77, 2010.
 [3] D. P. Bertsekas, R. G. Gallager, and P. Humblet, Data networks. PrenticeHall International New Jersey, 1992, vol. 2.
 [4] Y. Rekhter, T. Li, and S. Hares, “A border gateway protocol 4 (BGP4),” RFC 4271, 2006. [Online]. Available: http://www.ietf.org/rfc/rfc4271.txt
 [5] D. McPherson, V. Gill, D. Walton, and A. Retana, “Border gateway protocol (BGP) persistent route oscillation condition,” RFC 3345, 2002. [Online]. Available: http://www.ietf.org/rfc/rfc3345.txt
 [6] K. Varadhan, R. Govindan, and D. Estrin, “Persistent route oscillations in interdomain routing,” Computer networks, vol. 32, no. 1, pp. 1–16, 2000.
 [7] T. G. Griffin and G. Huston, “BGP wedgies,” RFC 4264, 2005. [Online]. Available: http://www.ietf.org/rfc/rfc4264.txt
 [8] G. Huston, “Interconnection, peering and settlements: Part I,” Internet Protocol Journal (Cisco), vol. 2, no. 1, June 1999.
 [9] ——, “Interconnection, peering and settlements: Part II,” Internet Protocol Journal (Cisco), vol. 2, no. 2, June 1999.
 [10] L. Gao and J. Rexford, “Stable internet routing without global coordination,” IEEE/ACM Transactions on Networking (TON), vol. 9, no. 6, pp. 681–692, 2001.
 [11] J. L. Sobrinho and T. G. Griffin, “Routing in equilibrium,” Mathematical Theory of Networks and System, 2010.
 [12] J. L. Sobrinho, “An algebraic theory of dynamic network routing,” IEEE/ACM Transactions on Networking (TON), vol. 13, no. 5, pp. 1160–1173, 2005.
 [13] C.K. Chau, “Policybased routing with nonstrict preferences,” in Proceedings of the 2006 conference on Applications, technologies, architectures, and protocols for computer communications, 2006, pp. 387–398.
 [14] J. L. Sobrinho and M. A. Ferreira, “Routing on multiple optimality criteria,” in Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication, 2020, pp. 211–225.
 [15] A. J. Gurney and T. G. Griffin, “Lexicographic products in metarouting,” in 2007 IEEE International Conference on Network Protocols. IEEE, 2007, pp. 113–122.
 [16] H. Geng, X. Shi, X. Yin, Z. Wang, and S. Yin, “Algebra and algorithms for multipath QoS routing in link state networks,” Journal of Communications and Networks, vol. 19, no. 2, pp. 189–200, 2017.
 [17] M. J. Fischer, N. A. Lynch, and M. S. Paterson, “Impossibility of distributed consensus with one faulty process,” Journal of the ACM (JACM), vol. 32, no. 2, pp. 374–382, 1985.
 [18] G. Klein, K. Elphinstone, G. Heiser, J. Andronick, D. Cock, P. Derrin, D. Elkaduwe, K. Engelhardt, R. Kolanski, M. Norrish, T. Sewell, H. Tuch, and S. Winwood, “seL4: Formal verification of an OS kernel,” in Proceedings of the ACM SIGOPS 22Nd Symposium on Operating Systems Principles (SOSP), 2009.
 [19] R. Kumar, M. O. Myreen, M. Norrish, and S. Owens, “CakeML: A Verified Implementation of ML,” in POPL, 2014.
 [20] X. Leroy, “Formal verification of a realistic compiler,” Commun. ACM, vol. 52, no. 7, pp. 107–115, 2009.
 [21] A. Guha, M. Reitblatt, and N. Foster, “Machineverified network controllers,” in PLDI, 2013.
 [22] A. Zaostrovnykh, S. Pirelli, L. Pedrosa, K. Argyraki, and G. Candea, “A formally verified NAT,” in SIGCOMM, 2017.
 [23] U. Norell, “Dependently typed programming in Agda,” in Proceedings of the 4th International Workshop on Types in Language Design and Implementation, 2009.
 [24] M. L. Daggitt, R. Zmigrod, L. v. d. Stoep, and T. G. Griffin, “Agda routing library,” 2021. [Online]. Available: https://github.com/MatthewDaggitt/agdarouting/tree/ton2021
 [25] M. L. Daggitt and T. G. Griffin, “Dynamic asynchronous iterations,” 2020.
 [26] M. L. Daggitt, A. J. Gurney, and T. G. Griffin, “Asynchronous convergence of policyrich distributed bellmanford routing protocols,” in Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, 2018, pp. 103–116.
 [27] M. L. Daggitt, “An algebraic perspective on the convergence of vectorbased routing protocols,” Ph.D. dissertation, University of Cambridge, 2019. [Online]. Available: https://www.repository.cam.ac.uk/handle/1810/292067
 [28] A. Üresin and M. Dubois, “Parallel asynchronous algorithms for discrete data,” Journal of the ACM (JACM), vol. 37, no. 3, pp. 588–606, 1990.
 [29] B. A. Carré, “An algebra for network routing problems,” IMA Journal of Applied Mathematics, vol. 7, no. 3, pp. 273–294, 1971.
 [30] J. Postel, “User datagram protocol,” RFC 768, 1980.
 [31] R. Chandra, P. Traina, and T. Li, “BGP communities attribute,” RFC 1997, 1996. [Online]. Available: http://www.ietf.org/rfc/rfc1997.txt
 [32] T. G. Griffin, “Exploring the stratified shortestpaths problem,” Networking Science, vol. 1, no. 1, pp. 2–14, Mar 2012. [Online]. Available: https://doi.org/10.1007/s1311901100036
 [33] T. G. Griffin and G. Wilfong, “Analysis of the MED oscillation problem in bgp,” in Network Protocols, 2002. Proceedings. 10th IEEE International Conference on. IEEE, 2002, pp. 90–99.
 [34] M. L. Daggitt and T. G. Griffin, “Rate of convergence of increasing pathvector routing protocols,” in 2018 IEEE 26th International Conference on Network Protocols (ICNP). IEEE, 2018, pp. 335–345.
 [35] P. Lahiri, G. Chen, P. Lapukhov, E. Nkposong, D. Maltz, R. Toomey, and L. Yuan, “Building scalable data centers: BGP is the better IGP!” presentation, 2012.
 [36] R. Beckett, R. Mahajan, T. Millstein, J. Padhye, and D. Walker, “Don’t mind the gap: Bridging networkwide objectives and devicelevel configurations,” in Proceedings of the 2016 ACM SIGCOMM Conference, 2016.
 [37] A. J. T. Gurney, “Asynchronous iterations in ultrametric spaces,” Tech. Rep., 2017. [Online]. Available: https://arxiv.org/abs/1701.07434
Appendix A Notational glossary
Notation  Meaning  Defined in 

Set of all node labels  Section IIB  
Set of all edges  Section IIB  
Set of all simple paths  Section IIB  
The invalid path  Section IIB  
Arbitrary origin node  N/A  
Arbitrary destination node  N/A  
Arbitrary neighbour node  N/A  
Arbitrary path  N/A  
Set of pathweights  Definition 1  
Choice operator over pathweights  Definition 1  
Set of policy functions for edge  Definition 1  
Set of all policy functions  Definition 1  
Trivial pathweight  Definition 1  
Invalid pathweight  Definition 1  
Function from pathweights to paths  Definition 2  
The constantly invalid policy  Section IID  
Total order over pathweights  Definition 7  
Strict total order over pathweights  Definition 7  
Arbitrary pathweights  N/A  
Arbitrary policy function  N/A  
Set of labels of nodes that participate  Section IID  
Set of routing tables ( vectors over )  Section IID  
Set of states ( matrices over )  Section IID  
Arbitrary adjacency matrix  Section IID  
Identity matrix  Section IID  
A single sync. iteration with topology  Section IID  
Synchronous state function  Definition 3  
Arbitrary routing tables  N/A  
Arbitrary routing states  N/A  
Arbitrary stable state  N/A  
The set of times of events of interest  Section IIE  
The set of epochs  Section IIE  
An arbitrary network  Section IIE  
Schedule activation function  Definition 4  
Schedule data flow function  Definition 4  
Schedule epoch function  Definition 4  
Schedule participants function  Definition 4  
Participants function  Section IIE  
Participation adjacency matrix  Definition 5  
Asynchronous state function  Definition 6  
Set of accordant states  Definition 12  
Set of all configurations  Section IIIB  
Set of free configurations  Section IIID  
Arbitrary time  N/A  
Arbitrary epoch  N/A  
Arbitrary set of participants  N/A  
Set of assignments  Definition 8  
Strict partial order over assignments  Definition 9  
Extended by relation over assignments  Definition 10  
Threatens relation over assignments  Definition 11  
Arbitrary assignments  N/A  
Dislodging order over assignments  Appendix B  
Height function over assignments  Appendix B  
Distance function over pathweights  Appendix B  
Distance function over routing table states  Appendix B  
Distance function over routing states  Appendix B 
Appendix B Proof of Theorem 2  Distance vector protocols
This appendix contains the proof that distancevector protocols over a finite routing algebra are convergent whenever the network topology is free.
Assume that we have a finite routing algebra and some network . We will prove that asynchronous model of the protocol is convergent whenever the network topology is free, by showing that the set of synchronous iterations is an AMCO over . The required result then follows by Theorem 1.
Ba Constructing the distance functions
Consider an arbitrary configuration consisting of an epoch and set of participants such that the participation topology is free. To fulfil the AMCO conditions, for each router we must construct a suitable distance functions, , that can be used to compare the different states of its routing table. Intuitively, our notion of distance will be based upon the ability of the router ’s current entries of dislodging entries in other router’s routing tables.
We now describe the formal construction of such a distance function. Note that for notational convenience, whenever we refer to a concept that depends an adjacency matrix we will use as a shorthand for . For example we will write instead of and instead of .
BA1 Dislodgement order over assignments
To begin with we construct a strict partial order over the set of assignments, which we will call the dislodgement order. The key idea for this is adapted from Sobrinho’s notion of a couplet digraph which we generalise to an assignment digraph. The vertices of this digraph are the set of assignments . There is an edge from assignment to assignment in the digraph if, in the current topology, either the latter is an extension of the former (i.e. ), or the former is strictly preferred to the latter (i.e. ). Then if there exists a path from assignment to assignment in this digraph.
Mathematically this relation is the transitive closure of the union of the and relations:
Intuitively, describes a chain of events triggered by a router choosing to use assignment that would eventually cause another router to remove assignment from its routing table, i.e. can eventually cause to be dislodged. In particular, each link represents an extension event and each link represents a choice event.
The crucial properties of this relation are as follows:

[label=O0]

is irreflexive  An assignment cannot dislodge itself.

is transitive  If assignment can dislodge and can dislodge then can dislodge .

respects  If assignment is extended by assignment then can dislodge .

respects  If assignment is strictly preferred to assignment then can dislodge .
Properties O2, O3 and O4 are immediate from the definition of . In contrast O1, the proof of irreflexivity, is nontrivial and uses the assumption that is free. However, modulo the differences in the algebraic structures and the change from couplets to assignments, the proof is identical to that of Lemma 1 in the appendix of [12]. Readers interested in a proof using our algebraic model may consult our accompanying Agda code.
BA2 Height of assignments
As both and are finite, the set of assignments is also finite. Therefore all upwards closed subsets under the relation must also be finite. Consequently the height of an assignment can be defined as the number of assignments it is capable of dislodging:
The crucial properties of are as follows:

[label=H0]

If is extended by then the height of is strictly less than the height of :

If is strictly preferred to then the height of is strictly less than the height of :
Comments
There are no comments yet.