Segment Routing: A comprehensive survey of research activities, standardization efforts and implementation results

In this paper we present a comprehensive survey on SR technology analyzing research activities, standardization efforts and implementation results. Firstly, we introduce the motivation for Segment Routing and its evolution from its public first in 2012. Then, we provide a common ground and a reference conceptual framework for the survey using a bottom-up approach. Next, we analyze research activities using a categorization approach. We have identified 7 main categories during our analysis of the current state of the art. They are respectively Monitoring, Traffic Engineering, Failure Recovery, Centrally Controlled Architectures, Path Encoding, Network Programming and Miscellaneous. We also look to the path of the standardization efforts, we describe the main 9 standardization efforts and we list other ongoing activities and provide pointers to them. Last but not least, we elaborate on the implementation results and we provide references for tutorials, source code and ready-to-go virtual setups.

READ FULL TEXT VIEW PDF

Authors

page 1

page 4

12/29/2020

A Survey on Segment Routing with Emphasis on Use Cases in Large Provider Networks

Segment routing is heralded as important technology innovation in large ...
07/23/2020

Micro SIDs: a solution for Efficient Representation of Segment IDs in SRv6 Networks

The Segment Routing (SR) architecture is based on loose source routing. ...
01/03/2019

An Efficient Linux Kernel Implementation of Service Function Chaining for legacy VNFs based on IPv6 Segment Routing

We consider the IPv6 Segment Routing (SRv6) technology for Service Funct...
04/23/2020

Implementation of Accurate Per-Flow Packet Loss Monitoring in Segment Routing over IPv6 Networks

Segment Routing over IPv6 (SRv6 in short) is a networking solution for I...
07/04/2018

Mapping the research software sustainability space

A growing number of largely uncoordinated initiatives focus on research ...
05/21/2019

TSN Algorithms for Large Scale Networks: A Survey and Conceptual Comparison

This paper provides a comprehensive survey of queueing and scheduling me...
10/24/2018

Flexible failure detection and fast reroute using eBPF and SRv6

Segment Routing is a modern variant of source routing that is being grad...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Segment Routing (SR) is based on the loose Source Routing concept. A node can include an ordered list of instructions in the packet headers. These instructions steer the forwarding and the processing of the packet along its path in the network. The single instructions are called segments, a sequence of instructions can be referred to as a segment list or as an SR Policy . Each segment can enforce a topological requirement (e.g. pass through a node or an interface) or a service requirement (e.g. execute an operation on the packet). The term segment refers to the fact that a network path towards a destination can be split in segments by adding intermediate way-points. The segment list can be included by the original source of the packet or by an intermediate node. When the segment list is inserted by an intermediate node, it can be removed by another node along the path of the packet, supporting the concept of tunneling through an SR domain from an ingress node to an egress node.

The research and standardization activities on Segment Routing originated in the late 2000’s, mainly with the goal of overcoming some scalability issues and limitations [1] that had been identified in the traffic engineered Multi Protocol Label Switching (MPLS-TE) solutions used for IP backbones. In particular it was observed that MPLS-TE requires explicit state to be maintained at all hops along an MPLS path and this may lead to scalability problems in the control-plane and in the data-plane. Moreover, the per-connection traffic steering model of MPLS-TE does not easily exploit the load balancing offered by Equal Cost MultiPath (ECMP) routing in plain IP networks. On the other hand, Segment Routing can steer traffic flows along traffic engineered paths with no per-flow state in the nodes along the path and exploiting ECMP routing within each segment.

In the early 2010’s, the IETF started the “Source Packet Routing in Networking” Working Group (SPRING WG) to deal with Segment Routing. The activity of the SPRING WG has included the identification of Use Cases and Requirements for Segment Routing (for example, [2], [3] and [4] have become IETF RFCs). Recently, the WG has issued the “Segment Routing Architecture” document (RFC 8402 [5]), while several other documents are still under discussion by the WG, as it will be analyzed later in this paper.

The implementation of the Segment Routing Architecture requires a Data Plane which is able to carry the segment lists in the packet headers and to properly process them. Control Plane operations complement the Data Plane functionality, allowing to allocate segments (i.e. associate a segment identifier to a specific instruction in a node) and to distribute the segment identifiers within an SR domain.

As for the Data Plane, two instantiations of the SR Architecture have been designed and implemented, SR over MPLS (SR-MPLS) and SR over IPv6 (SRv6). With SR-MPLS, no change to the MPLS forwarding plane is needed [6]. SRv6 is based on a new type of IPv6 routing header called SR Header (SRH) [7]. Temporally, SR-MPLS has been the first instantiation of the SR Architecture, while the recent interest and developments are focusing on SRv6. In particular, the IPv6 SR dataplane is being extended to support the so-called SRv6 Network Programming Model [8]. According to this model, segment routing functions can be combined to achieve an end-to-end (or edge-to-edge) networking objective that can be arbitrarily complex. This is appealing for implementing complex services like Service Function Chaining. SRv6 can be used as an overlay tunneling mechanism directly exposed and used by servers (similar to VXLAN tunneling) and as a transport mechanism in network backbones (supporting Traffic Engineering and Resilience services). In this vision, SRv6 can simplify network architectures avoiding the use of different protocol layers.

As for the SR Control Plane operations, they can be based on a distributed, centralized or hybrid approach. In the distributed approach, the routing protocols are used to signal the allocation of segments and the nodes take independent decisions to associate packets to the segment lists. In the centralized approach, an SR controller allocates the segments, takes the decision on which packets needs to be associated to which segment lists and configures the nodes accordingly. Very often, an hybrid approach which consists in the combination of distributed and centralized approach is used.

The goal of this paper is to provide a comprehensive survey on the Segment Routing technology, including all the achieved results and the ongoing work. We will consider the standardization efforts (section IV), the research activities (section III), the implementation results (section V) and ongoing deployments (section VI). In addition, in section II we provide a short introduction to the main concepts of the Segment Routing architecture, while we highlight future research directions and open issues in section VII.

To the best of our knowledge, there is only another survey about Segment Routing, namely [9]. With respect to [9], our survey is more complete in the analysis of the scientific literature, covering more than 75 papers. Moreover, it provides a classification and discussion of standardization activities, and considers the status of implementations (the analysis of standardization efforts and implementation results are both missing in [9]).

Ii The Segment Routing (SR) architecture

This section includes a short introduction to the main SR architectural aspects. Our goal is to provide a common ground and a reference conceptual framework for the survey, rather than to cover the details in a tutorial way.

The seminal paper on the Segment Routing Architecture is [10]. Published in 2014, it provides an overview of the motivations for SR, describes a set of important use cases and illustrates the architecture. The basic concepts proposed in [10] have been elaborated and refined in the RFC 8402 [5] which has recently completed its standardization process in the IETF (July 2018). Obviously, RFC 8402 [5] represents the most important source of information for the SR architecture. The work in [11] (published in 2017) provides a short and effective introduction to the Segment Routing architecture, with focus on the MPLS dataplane. The survey paper [9] has a section about the SR architecture, which tries to give more details related to both the Data Plane and the Control Plane aspects.

Following the RFC 8402, let us start by discussing the general concepts of SR, which are independent from the specific dataplane (MPLS or IPv6). An SR policy is an ordered list of segments (segment list). As shown in Fig. 1, the segment list is added to the packet headers by a headend node that steers the packets of a flow onto the SR policy. A Segment Routing domain (SR domain) is the set of nodes participating in the source-based routing model. The headend node can be the originator of the packet or (as Fig. 1) an intermediate node that performs a classification of the traffic and associates the SR policies to the packets. In other words, hosts can be part of an SR domain, but this is not required and depends on the overall scenario in which SR is applied. It is expected that all nodes in an SR domain are managed by the same administrative entity. For example, a Service Provider backbone can constitute an SR domain and the headend node will be the ingress edge router of the backbone (in this case hosts are not part of the SR domain).

A segment is described by a Segment Identifier (Segment ID or SID). For the SR-MPLS dataplane, a SID is an MPLS label, while for the SR-IPv6 dataplane a SID is an IPv6 address. An SR policy P consisting in steering a packet along three segments with SIDs S1, S2 and S3 can be represented as P=<S1,S2,S3> (see Fig. 1). Three basic operations on SIDs and segment lists have been defined for a generic SR dataplane: PUSH, NEXT and CONTINUE. We assume for simplicity that S1, S2 and S3 represent topological instructions, so that the policy P instructs the packet to cross three nodes in sequence, identified by the SIDs S1, S2 and S3. The PUSH operation consists in the insertion of a segment on top of the segment list, i.e. as the new first segment of the SR policy. In order to build the SR policy P described above, the headend node executes the PUSH operations in this order: PUSH(S3), PUSH(S2), PUSH(S1). In an SR packet, the segment that specifies the instruction to be executed is called active segment. In the considered example with the SR policy P, the headend node will send the packet with active segment S1. The NEXT operation is executed by a node that has processed the active segment and considers the next segment of the SR policy to be executed. In our example, the node identified with SID S1 receives the packet and perform the NEXT operation. The next segment is S2, which becomes the active segment and the packet is forwarded toward S2. The NEXT operation also covers the case of the last node of an SR policy, in which the NEXT operation usually results in processing the packet according to regular IP forwarding. The CONTINUE operation is performed by nodes that are in the path between two segments. For example, the intermediate nodes in the path between S1 and S2 perform the CONTINUE operation. The path between S1 and S2 is not prescribed by the SR policy and will be chosen considering the regular IP routing toward node S2 in the SR domain. If there are multiple equal cost paths between nodes S1 and S2 (as in Fig. 1) and the ECMP (Equal Cost MultiPath) mechanism is supported by the IP routing in the SR domain, it can be conveniently exploited by Segment Routing.

Fig. 1: Example of an SR policy and of SR operations

The segments can be classified into

Global Segments and Local Segments. Global Segments correspond to instructions that are globally valid in an SR domain. Local segments correspond to instructions that are valid within a single node. The typical example of a global segment is an instruction to forward packets towards a given destination IP network or a destination IP node. Considering that an IGP (Interior Gateway Protocol) routing protocol (e.g. OSPF or ISIS) is used in the SR domain, these instructions are called IGP-prefix segment and IGP-node segment (or simply prefix segment and node segment). All nodes in the SR domain can execute the prefix segment or node segment instructions by considering the path towards the destination network or destination node in their own routing table. The most important example of local segment is the instruction to forward a packet to a node identified as adjacent by the IGP routing protocol. This corresponds to sending the packet on a specific outgoing interface and can be executed only on a specific node. This instruction is called IGP-Adjacency Segment. Thanks to the use of IGP-Adjacency segments, it is possible to prove that any path across an SR domain can be expressed by an SR Policy (which can include a combination of global and local segments). Local segments can also be used to represent service instructions to be executed in a given node. The mapping of global and local segment into Segment Identifiers (SIDs) and the distribution of the SIDs in an SR domain are specific to the two different dataplanes (MPLS and IPv6) and will be discussed in the next subsections.

The IGP-Anycast Segment is an IGP-Prefix segment that corresponds to an anycast prefix, i.e. a prefix advertised by a set of routers that can be used for High Availability or Load Balancing purposes.

The Binding Segment is used to associate an SR policy to a SID (called Binding SID or BSID). A packet received with the BSID will be steered on the associated SR policy, this means that the packet will be forwarded using the corresponding Segment List. Using the Binding Segment it is possible to separate the processes of packet classification by the enforcing of a specific SR Policy. The SR policy can be changed over time (and can be executed in a different node) with no need to change the classification process. This improves the scalability and service independence of the solutions based on Segment Routing.

Segment Routing is supported by two different dataplanes: i) MPLS and ii) IPv6. Table I summarizes the mapping of the SR concepts into the two dataplanes and will be discussed in the next two subsections.

Generic SR SR-MPLS SRv6
SR Policy Label Stack
Segment List (of IPv6
addresses) in the SR Header
Active Segment Topmost Label
IPv6 address indicated
by the Segment Left field
PUSH
Operation
Label Push
Adding an IPv6 in the Segment
List in the SR Header
NEXT
Operation
Label POP
Decrementing the Segment Left
field, copying the active segment
in the IPv6 Destination Address
CONTINUE
Operation
Label Swap
Forwarding according to IPv6
Destination Address
TABLE I:
Mapping SR concepts into SR-MPLS and SRv6

Ii-a MPLS dataplane (SR-MPLS)

The MPLS dataplane (SR-MPLS) is specified in [6]. For SR-MPLS, Segment Routing does not require any change to the MPLS forwarding plane. An SR Policy is instantiated through the MPLS Label Stack: the Segment IDs (SIDs) of a Segment List are inserted as MPLS Labels. The classical forwarding functions available for MPLS networks allow implementing the SR operations. The PUSH operation corresponds to the Label Push function, i.e. pushing one or more labels on top of an incoming packet and then sending it out of a particular outgoing interface. The NEXT operation corresponds to the Label Pop function, i.e. removing the topmost label. The CONTINUE operation corresponds to Label Swap function, i.e. removing the incoming label and inserting an outgoing one. The encapsulation of an IP packet into a SR-MPLS packet is performed at the edge of an SR-MPLS domain if the destination address of the packet matches a pre-configured Forwarding Equivalent Class (FEC) associated with a specific SR Policy.

The mapping of Segments to MPLS Labels (SIDs) is a critical process in the SR-MPLS dataplane. In the general case, different routers in the SR domain could have different available ranges of labels to be used for Segment Routing. Therefore each router can advertise its own available label space to be used for Global Segments called SRGB - Segment Routing Global Block (in general, this label space can even be composed of a set of non contiguous blocks). For this reason, in the SR domain the Global Segments are identified by an index, which has to be re-mapped into a label taking into account the node that will process the label. Assuming that the SRGB of a node is a label range starting from 10000, for a Global Segment with index X, the node needs to receive the label 10000+X. As an example, in Fig. 2 A we consider how to implement the SR policy described in Fig. 1 using the SR-MPLS dataplane in the general case in which different nodes are using different SRGBs. The SRGBs of the nodes and the segment index associated to the segments S1, S2 and S3 are shown in the gray rectangle. The headend node needs to consider in advance which is the SRGB of the nodes that will perform the NEXT operation the segments, because the label for the next segments needs to be crafted accordingly. In particular, the initial label for segment S2 set by the headend node will be 1002, i.e. the SRGB of node S1 (1000) plus the index for segment S2 (2). Node S1 will have to modify the label to 4002 if the packet is forwarded to node N4 (whose SRGB is 4000) or to label 6002 if the packet is forwarded to node N6 (whose SRGB is 6000). Both nodes N4 and N6 will remap (swap) the label to 2002 when forwarding the packets to S2. The initial label for node S3 set by the headend node is 2003, i.e. the SRGB of node S2 (2000) plus the index for segment S3 (3). This label will reach node S2 unmodified, then it will be properly processed by node S2 that will remap (swap) it considering the SRGB of the next hop in the path towards node S3. This remapping process complicates the operations and the troubleshooting. There are also services (e.g. involving anycast segments) that cannot be realized if different SRGBs are used by different nodes. For this reason, [5] strongly recommends that an identical range of labels (SRGB) is used in all routers, so that a Global Segment will always be mapped to the same SID (MPLS label) in all nodes. In Fig 2 B we present the mapping of the same SR policy described in Fig. 1 under the suggested operating mode in which an identical SRGB is used in all nodes. We observe that the MPLS labels do not need to be remapped, so that the same label consistently identifies the same segment throughout the SR domain.

Fig. 2: SR-MPLS dataplane: mapping segments to labels using the SRGB

Ii-B IPv6 dataplane (SRv6)

For the IPv6 dataplane (SRv6), a new type of IPv6 Routing Extension Header, called Segment Routing Header (SRH) has been defined in [7]. The format of the SRH is shown in Fig. 3. The SRH contains the Segment List (SR Policy) as an ordered list of IPv6 addresses: each address in the list is a SID. A dedicated field, referred to as Segments Left, is used to maintain the pointer to the active SID of the Segment List. To explain the SRv6 dataplane, we consider three categories of nodes: Source SR nodes, Transit nodes and SR Segment Endpoint nodes. A Source SR node corresponds to the headend node discussed above. It can be a host originating an IPv6 packet, or an SR domain ingress router encapsulating a received packet in an outer IPv6 header. In Fig. 4 we consider the latter case, the Source SR node is an edge router that encapsulates a packet (which can be IPv6, IPv6 or even a Layer 3 frame) into an outer IPv6 packet and inserts the SR Header (SRH) as a Routing Extension Header in the outer IPv6 header. The encapsulated packet is indicated as Payload in Fig. 4. The Segment List in the SRH is composed of S1, S2 and S3 which are stored in reverse order (the fist SID is S3, the last segment in the SR policy). The Segment Left field is set to 2, so that the active segment is S1, represented in red in the figure. The Source SR node sets the first SID of the SR Policy (S1) as IPv6 destination address of the packet. These operations correspond to a sequence of the PUSH operations described above. The SR Segment Endpoint node receives packets whose IPv6 destination address is a local address of an interface or is locally configured as a segment. The SR Segment Endpoint node inspects the SR header: it detects the new active segment, i.e. the next segment in the Segment List, modifies the IPv6 destination address of the outer IPv6 header and forwards the packet on the basis of the IPv6 forwarding table. These operations correspond to the NEXT operation described above. In Fig. 4, we can see that S1 is the first SR Endpoint node, it decrements the Segment Left fields to 1, making S2 the active segment, and sets S2 as IPv6 Destination Address. A Transit node forwards the packet containing the SR header as a normal IPv6 packet, i.e. on the basis of the (outer) IPv6 destination address, because the IPv6 destination address is not a local address, nor it has been configured as a segment. This behavior corresponds to the CONTINUE operation. In Fig. 4, nodes N4, N5, N6 and N7 are Transit nodes, which perform a regular forwarding of the packet towards the IPv6 Destination Address.

Fig. 3: Segment Routing Header
Fig. 4: SRv6 dataplane operations

In the given example, the PUSH operation is performed by encapsulating a packet (IPv6, IPv4 or Layer 2 frame) into an outer IPv6 packet with a Segment Routing Header. Another possibility is to perform the insertion of an SRH as a new header between the IPv6 header and the Next Header (e.g. the Trasport Layer Header, TCP or UDP), without encapsulating the packet in a new IPv6 packet. This option only applies to IPv6 packets and it is especially suited in case a host is acting as Source SR node (Headend node).

In addition to the basic operations (PUSH/ NEXT/ CONTINUE), the SRv6 Network Programming model [8] describes a set of functions that can be associated to segments and executed in a given SRv6 node. Examples of such functions are: different types of packet encapsulation (e.g. IPv6 in IPv6, IPv4 in IPv6, Ethernet in IPv6), corresponding decapsulation, lookup operation on a specific routing table (e.g. to support VPNs). A more complete list is provided in [8], but the list is not meant to be exhaustive, as any function can be associated to a segment identifier in a node. Obviously, the definition of a standardized set of segment routing functions facilitates the deployment of SR domains with interoperable equipment from multiple vendors.

According to [8], we can revisit the notion of Segment IDentifier (SID) taking into account that IPv6 addresses are used as SIDs in SRv6. A 128 bit SID can be logically split in two fields and interpreted as LOCATOR:FUNCTION where LOCATOR includes the L most significant bits and FUNCTION the remaining 128-L least significant bits. The locator corresponds to an IPv6 prefix (e.g. with a length of 48 bits) that can be distributed by the routing protocols and provides the reachability of a node that hosts a number of functions. The length of the locator is not fixed and can be chosen by each operator for its own SR domain (also independently for different nodes). All the different functions residing in a node can share the same locator and have a different FUNCTION code, so that their SID will be different. From the routing point of view, the solution is very scalable as a single prefix is distributed, with limited impact on the routing tables of the nodes in the SR domain. The LOCATOR:FUNCTION representation of a SID can also be extended by splitting the FUNCTION field into FUNCTION:ARGUMENTS, so that the least significant bits can be used to provide information to a given function.

The concept of Network Programming consists in combining functions that can reside in different nodes to achieve a networking objective that goes beyond mere packet routing [8]. The functions described in [8] can support valuable services and features like for example layer 3 and layer 2 VPNs, traffic engineering, fast rerouting. The Network Programming model offers the possibility to implement virtually any service by combining the basic functions in a network program that can be embedded in the packet header. As shown in Fig. 3, the SRH can include an optional section that carries Type Length Value (TLV) objects. These TLV objects can be defined to carry information that needs to be elaborated by one or more segments of an SR policy (Segment List). For example, the so-called HMAC TLV can be added and used to verify that an SRH header has been created by an authorized node and that the segment list is not modified in transit. Another potential use of TLV objects is for exchanging Operation and Maintenance (OAM) information among the nodes of the SR domain.

Ii-C Control plane for SR and relation with SDN

Control Plane operations are needed to complement the dataplane functionality and provide a complete solution for Segment Routing. The Control Plane can be based on a fully distributed approach, in which the routers are capable to take independent decisions to setup and enforce the SR Policies, it can rely on a centralized SR controller that takes decision and instructs the routers following the SDN principles, or on an combination of the two approaches (hybrid solution).

For the SR-MPLS dataplane, the definition of a fully distributed approach has been worked out within the IETF, with the definition of extensions to the IGP routing protocols (OSPF, ISIS, see [12] [13] [14]). These extensions to the routing protocols are used by each routers to advertise the different types of IGP-segments (prefix, node, adjacency, anycast) and to distribute some SR configuration information. All other routers in the SR domain will receive this information by means of the IGP routing protocol. This information is needed to map the segments included in an SR policy into SIDs represented as MPLS labels in the SR-MPLS dataplane. As we have discussed in subsection II-A, in the general case each router could allocate different ranges of labels to be used for Global Segments. The range of labels used for the global segments by a router, called SRGB - Segment Routing Global Block is among the SR configuration information advertised using the routing protocol. We recall that it is strongly recommended to use an identical range of labels (SRGB) in all routers.

For the IPv6 dataplane, the process of advertising the IGP-prefix, IGP-node and IGP-anycast segments is simplified thanks to the use of IPv6 addresses as SIDs. In particular, there is no need to extend the IGP routing protocols to distribute these segment types, represented as IPv6 prefixes that are natively distributed by the routing protocols. Also the definition of a Segment Routing Global Block as in the SR-MPLS is not needed and the operations related to Global Segments can rely on IPv6 addresses that are globally routable in the SR domain. This means that the Control Plane for SRv6 can use the regular IGP routing protocols (OSPF, ISIS) to support the basic operations, while extensions are still needed ([15] [16]) to distribute IGP-Adjacency segments and other SR configuration information.

The definition of the control plane for Segment Routing has started from the SR-MPLS dataplane and then the SRv6 dataplane has inherited most of the functionality, which has been adapted to the new dataplane. We observe that an original design goal of the control plane for Segment Routing has been to support the fully distributed approach, in which routers are capable to take autonomous decisions. This allows offering the the same functionality of a traditional MPLS network, which does not need a centralized SDN controller for its operations. On the other hand, we observe now a trend to focus on an hybrid approach, in which distributed routing protocols coexist with an SR controller. This hybrid approach is aligned with the vision of Software Defined Networking that aims at removing complexity from distributed nodes and to centralize control plane function in SDN controllers. In this light, the Segment Routing architecture can be deployed by seeking the right balance between distributed and centralized control. The distributed control is used by the routers to exchange reachability information and evaluate the shortest paths in a traditional way, with no need to interact with the centralized controller. We observe that this is the best approach to provide connectivity in Wide Area Networks in which the control connections between the nodes and the SDN controllers are affected by non-negligible latency and failure probability. Segment Routing can be used for Fast Reroute, by pre-configuring SR policies that provide alternate paths in case of link or node failures and are automatically activated by the node when the failure happens. The pre-calculation of such SR policies can be performed in a distributed mode or can be centralized in a controller. Basic topology information and additional information for Traffic Engineering need be conveyed to the controller, as well as service related information that is advertised by nodes using distributed routing protocols. The SDN controller can receive this information in different ways. For example, it can participate to the IGP routing protocol, it can interact with routers in a proprietary way to extract their IGP databases, it can receive information by routers using extensions to BGP-LS (BGP-Link State). Whatever mechanism has been used to retrieve the needed information from the nodes, the SDN controller is in charge to take decisions about the SR policies that implement advanced features or services like Traffic Engineering, VPNs or Service Function Chaining. This approach allows to clearly decouple the dataplane operations from the service logic operating in the control plane. The mechanisms and protocols for the SDN controller to enforce the SR policies by configuring the the nodes are left open in the SR architectural definition. As mentioned in

[5], some options are Network Configuration Protocol (NETCONF), Path Computation Element Communication Protocol (PCEP) [17], and BGP. The main characteristic of the Segment Routing solution compared to other SDN solutions (e.g. based on OpenFlow) is that only the edge nodes needs to be configured to enforce a given SR policy, while the internal nodes do not need to keep state per SR policy. This feature gives a substantial improvement in terms of scalability.

Ii-D Segment Routing motivations and use cases

As anticipated in the introduction section, the RFC 5439 [1] has identified some scalability issues of traditional MPLS networks with Traffic Engineering support. These issues originated the interest in defining a more scalable solution like Segment Routing back in the late ‘00s. Several use cases and requirements for Segment Routing has been collected in a number of documents. In [2], the main use cases identified are: MPLS tunneling (i.e. to support VPN services), Fast ReRoute (FRR), Traffic Engineering (further classified in a number of more specific use cases). A set of Resiliency use cases is described in [3]. In [4], the Segment Routing use cases for IPv6 networks are considered, with a set of exemplary deployment environments for SRv6: Small Office, Access Network, Data Center, Content Delivery Networks, Core Networks.

Iii Research activities

In this section, we describe the research activities on SR, and we provide two different classifications to characterize the research papers on the basis of their main scope. We also show how to extract useful information regarding the ongoing SR research activity from analyzing the relationship among the two classifications proposed.

The first classification proposed is based on the identification of seven main Research Categories, as reported in Table II. The first one is the Monitoring category, collecting all the works that describe and implement tools related to network monitoring activities. Some example are the measurements of the end to end delay over a given input route or the assessment of the volume of the traffic flows. The second category is Traffic Engineering, where we include all the works proposing advanced routing strategies to optimize the network performances. The third category is Failure Recovery, covering solutions to provide fast network recovery in the case of node/link failure; due to the time scale constraints, the works in this category are based on local mechanisms, i.e. not involving the central controller. The fourth category defined is Centrally Controlled Architectures, including all the papers focusing on the implementation of an SR network with a centralized control plane realized on top of an underlay network (IP, SDN, MPLS). Here we point out that, despite some of the works classified as Traffic Engineering are based on a centrally controlled architectures, they are not included in the Centrally Controlled Architectures category, since their main scope remains to optimize a TE goal (such the reduction of the congestion, or the minimization of the energy consumption). In the Path Encoding category we group all the papers that propose algorithms aiming at translating network paths into an SL; specifically, taking as input a path in the form of a sequence of nodes and links, the generic path encoding algorithm provides as output a sequence of SIDs to be pushed in the packet header, so that to steer the packet along the input path. The sixth category is Network Programming, where we inserted the scientific works that propose solutions that exploit the programmability feature of SR. i.e. using service based SIDs to define the functions to be executed on packets crossing a specific segment list. A significant example of works falling into this category, are all the ones related to Service Function Chaining. Finally, we define a Miscellaneous category, where we put all the works not belonging to previous categories.

Category References
Monitoring (MON) [18, 19, 20, 21, 22, 23, 24]
Traffic Engineering (TE) [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43]
Failure Recovery (FR) [44, 45, 46, 47, 48, 49, 50, 51, 52]
Centrally Controlled
Architectures (CCA)
[53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66]
Path Encoding (PE) [67, 68, 69, 70, 71, 72, 73, 74]
Network Programming (NP) [75, 76, 77, 78, 79, 80, 81, 82]
Miscellaneous (MISC) [83, 84, 85, 86, 87, 88]
TABLE II:
Classification #1 based on research categories
Fig. 5: Number of references for each category of the defined taxonomy.

In Fig. 5 we report an histogram showing the number of references falling into each of the defined categories. Analyzing Fig. 5, it is evident that Traffic Engineering and Centrally Controlled Architectures are the most investigated subjects, while the other categories have been covered by almost the same number of works. This behavior is quite expected since the main feature of SR is to define routing paths in a very flexible way and to make use of widely deployed data planes; this aspect is really appealing for the definition of new solution to TE problems and for the realization of overlay networks. On the contrary, the number of papers related to Network Programming could suggest a low interest of the research community to such a novel topic. We strongly believe that Network Programming represents a fruitful research topic for next future, and the low number of available papers is only due to its early definition state.

In order to better investigate the research activity related to SR, we propose a second classification based on the specific SR topics considered in the research works. The new SR-related classification is reported in Tab. III. The different SR topics are also aggregated in three main groups:

  • SR feature exploitation;

  • SR functions optimization;

  • SR extensions.

The SR feature exploitation group covers all the works making use of SR features to solve classical networking problems, such as network resource optimization and performance improvement. The first SR topic is the routing flexibility, i.e., the possibility of steering a packet over a non trivial path (e.g., containing ECMP, loops, ect.). The second feature is the source routing, i.e. the capability of SR to instruct only the source node for the configuration of a specific network path. The third SR feature is the programmability, i.e., the possibility to force a packet to go through a function by using specific SIDs. All the remaining SR features are included in the other topic. We merged here together the following topics: i) the Adj-SIDs, used to force a packet to be forwarded on a specific output port; ii) the ECMP, i.e. the ability of SR to balance the traffic over multiple paths provided by the IGP routing protocols; iii) the Type Length Value (TLV) used to add optional data to the SR header; iv) the Binding SID (BSID) that allows to define SR tunnels in a transit node; v) the SR traffic counters, which collect traffic statistics based on the active segment carried in the packets headers; and vi) the spray policy, which allows to duplicate an incoming packet over multiple output port and using different SLs.

In the SR functions optimization group, the research activities aiming at improving the inner functions of an SR network, are inserted. The first topic of the group is the SR Steering Policy, i.e. the definition of policies to be installed on the network devices in order to attach the proper SL to each incoming packet. The second topic is the SL length, i.e. the number of SIDs of a segment list, that has an impact on SR header insertion and processing.

The last group, i.e. SR extensions, is represented by a single topic (new functions) and is related to new functionality implemented in SR to support advanced services.

Tab. III reports the classification of the research works according to the SR topics. Differently than Tab. II, where the same reference can appear only once, in Tab. III, it can be classified under different categories. As a matter of example, [72] uses the Adj-SIDs SR feature in order to perform SR functions optimization from the point of view of SL length.

SR feature exploitation SR functions optimization Extend SR
Routing Flexibility Source Routing Programmability Others SR steering policy SL length New Functions
[19],[21],[22],[26]
[28],[29],[31],[34]
[35],[38],[43]
[18],[19],[20]
[25],[26],[27]
[30],[32],[33]
[34],[35],[37]
[39],[40],[41]
[42],[48],[53]
[54],[55],[56]
[57],[58],[59]
[60],[61],[62]
[64],[65],[66]
[24],[51],[75],[76]
[77],[78],[79],[80]
[81],[82]
BSID:[44],[52],[61],[62],[65]
ECMP:[32],[38],[39]
TLV:[24],[44],[51],[54]
Adj-SID:[37],[68],[70],[71],[72]
Spray:[45],[46]
Traffic counters:[23]
[27],[37],[35]
[73],[74]
[25],[30]
[37],[38]
[42],[43]
[67],[68]
[69],[70]
[71],[72]
[73],[74]
[24],[48]
[49],[50]
[71],[74]
[78],[81]
TABLE III:
Classification #2 based on SR topics

Tab. III shows that the most used SR feature is the source routing one, while there is still a limited amount of works focusing on network programmability and on the definitions of new functions.

Fig. 6: Graph showing the relation between the research categories and the SR topics.

In order to obtain further insights about the SR research activity, we defined a way to show the relationship between the classifications reported in Tab. II and Tab. III, respectively. In Fig. 6 we report a graph, defined in the following way: each node represents a research category (violet rectangles at the center of the figure) or an SR topic (divided in red, blue and green rectangles at the border of the figure), and an edge among a category and an SR topic is present only if both are present in the same work. Moreover, the thickness of an edge depends on the number of papers covering the specific category/topic pair. The graph reported in Fig. 6 shows several interesting outcome: i) the works related to the Monitoring category mainly exploit the routing flexibility and the source routing paradigm; ii) a same result is obtained for the Traffic Engineering works, but in this case also the SR steering policy and SL length topics are covered; iii) in order to provide Failure Recovery solutions, new functions generally need to be defined; iv) the source routing is the most used SR feature for Centrally Controlled Architecture solutions, since it allows to reduce the communication between the central controller and the network devices; v) as expected, the Path Encoding works are mainly focused on the optimization of the SL length (it is anyway interesting to notice that the Adj-SID is the main SR feature used to get this scope); vi) finally, works falling in the Network Programming category always exploit the programmability feature of SR, and, in some cases, new functions need to be defined.

Fig. 7: Histogram showing the relation between the Classifications defined in Tab. II and Tab. III.

In Fig. 6, we report for each category/SR topic the number of related references.

In the next subsections we briefly describe all the research works, considering the classification proposed in Tab. II.

Iii-a Monitoring

Seven research works proposing monitoring solutions able to exploit SR have been defined, as shown in Table IV. These works have been classified on the basis of their main aim:

  • delay measurements, aiming at obtaining the delay for links and routers exploiting the possibility of modifying the SL at source nodes, i.e. source routing SR feature;

  • health checking of network devices, aiming at monitoring the network state exploiting the capability of SR in defining ad-hoc routes, i.e. routing flexibility feature.

  • traffic measurements, aiming at assess the traffic matrix of a network exploiting routing flexibility and SR counters;

  • traceroute, aiming at implementing the well known traceroute utility in SR.

Objective References
Delay measurements
[18],[24]
Health checking
of network devices
[19],[20]
Traffic measurements [21],[22],[23]
Traceroute [24]
TABLE IV:
Classification of the references related to Monitoring.

In the following we provide a brief overview of the references classified as monitoring related works.

[18] proposes a novel monitoring system powered by Segment Routing (SR) which is used for the provisioning of delay-aware network services spanning multiple-domains. Based on SR-MPLS principles, it enables the delay measurements over multiple candidate routes without requiring related LSP signalling sessions. Authors consider two types of probes using SR-MPLS. The first type is originated and terminated by network stations and allows to retrieve only round-trip measurements and they have less accuracy. Moreover they are typically used for performance measurements over a single link. Instead, the second type relies on external monitoring components which inject and receive timestamped probes routed according to the enforced SR segment list. The second type of measurements requires synchronization between the end-points but allows to measure also unidirectional delay which are more useful when it is necessary to deploy services in the network.

The traffic steering capabilities of SR have been used in SCMon [19], a new solution for continuous monitoring of the data-plane. It allows to track the health of all routers and links: i) forcing “monitoring probes to travel over cycles”; and (ii) testing “parallel links and bundles at a per-link granularity”. The key insight is that network nodes compute a second network graph and calculate routes on this monitoring topology which spans all network links. Then, nodes carefully select ECMP paths and enforce packet forwarding through cycles leveraging SR in order to detect/localize failures and overloading of single/multiple links. A prototype implementation of SCMon has been evaluated on publicly available topologies and emulated networks. In the first experiment the ratio number of cycles over number of edges is evaluated to analyze the percentage of the networks covered. Then the authors evaluate the time to detect black holes showing that most of them are detected within less than 100 msec.

In [20], authors focus on bandwidth-efficient monitoring schemes based on cycles. They propose four different algorithms to compute cycles which are designed to traverse/cover every link in the network. This optimization based on cycles allows to save network resources and to monitor the network from a single point of advantage. Segment Routing is used as transport technology to forward the probes along the network. The paper builds upon the results of [19]: authors leverage the phase 1 of SCMon to build the monitoring topology of the network and then apply their two-phase algorithms in order to minimize the cycle cover length. Performance evaluation shows the effectiveness of these algorithms in terms of cycle cover length and segments list depth and the improvements respect to the baseline (SCMon).

[21] exploits the flexibility of SR to perform traffic measurements and get the Traffic Matrix. A traffic measurement is performed by rerouting a flow and checking the load variations caused on the network links. Even though the idea of measuring traffic through routing perturbations is not new, SR turns out to be an enabling technology for the applicability of such an approach. In fact, while in the past traffic flows were rerouted by acting on the OSPF link weights, causing routing instability and performance degradation, SR allows to modify a path by simply acting on the ingress node, then reducing the impact of a rerouting. In [21]

, the problem of assessing the TM while minimizing the routing perturbations is formulated as an ILP and an heuristic algorithm called SERPENT is presented. Due to the high computational complexity of SERPENT, a lighter greedy heuristic called PaCoB is proposed in

[22].

An attractive feature of SR is the introduction of specific interface counters that allow to get statistics on network traffic flows. The update of the traffic counters is related to the normal processing performed at the SR data plane, thus having a negligible impact on the router performance. The simplest type of SR counters, named Base Pcounters, collect traffic statistics (byte/packets) passing through a router and having a specific active segment. Enhanced SR counters, named TM Pcounters, allow to distinguish between traffic that is internal to the SR domain and flows that are injected into the SR domain. Specifically, a TM Pcounter collects traffic statistics of traffic flows received by an interface marked as external (this is a configuration option for the network operator). Starting from the availability of this new type of traffic related information, in [23] the Traffic Matrix Assessment problem has been extended to include the SR counters measurements. Authors show that, depending on the structure of the SLs used in the network, the TM can be assessed with no error.

[89] extends the Linux kernel to run eBPF programs as in-network functions attached to the SRv6 SIDs; further details about the implementation are provided in the Section V. The authors demonstrate the effectiveness of their approach building three different applications. The first one realizes a passive monitoring solution of network delays (direct links or end-to-end paths) [24]. The high level idea is that a small percentage of traffic is encapsulated with a special SRH carrying on additional information like timestamps. These timestamps are then used by the recipient nodes to calculate one-way or round-trip delays. The second application realizes a link aggregation group using SRv6. In particular a weight-round-robin scheduling is realized to aggregate the bandwidth of two different links. Finally, an enhanced version of traceroute has been realized implementing a new SRv6 behavior, the so called End.OAMP. This behavior when triggered performs a fib lookup in the node and return to a destination address specified in the SRH all the ECMP next hops. If possible this function is leveraged at each hop, otherwise the program falls back to the legacy ICMPv6 mechanism.

Iii-B Traffic Engineering

Due to its appealing features in terms of routing flexibility, SR is widely used to face Traffic Engineering related problems. During our investigation, we have found nineteen papers exploiting SR to provide advanced TE solutions. The TE research works have all the classical structure of an optimization problem: i) an objective function must be minimized/maximized, ii) taking into account a set of parameters, and iii) considering a specific network scenario. Three different TE objectives have been covered by the literature, i.e., the minimization of the network energy consumption, the optimization of the congestion and the minimization of the number of rejected requests. The high routing flexibility allowed by SR might cause the presence of complex and long network paths. For this reason, while optimizing the routing according to the specific objective, it is important to take into account the impact that excessively complex routing solutions might have on the network performance. Further than the end to end delay, some of the reviewed works also take into account SR related overhead, both in terms of bandwidth wasted due to the insertion in the packets of the SR header, and the number of SR steering policy to be configured in the edge routers. Finally, the considered network can be a full SR one, i.e., all the nodes are SR capable, or a partially deployed SR, where only a subset of nodes can process the SR header.

Table V show the classification of the TE related references. It is interesting to notice that most of the works consider the congestion minimization as main objective, and that there are few solutions that can work also in a hybrid network scenario.

Objective
Energy
Consumption
[28],[29]
Link Bandwidth
[25], [26],[27],[30],[31]
[32],[33],[34],[35],[37]
[38],[39],[40],[41],[42],[43]
Rejected Requests [25],[34]
Takes into
account
Delay [26],[34],[42],[43]
SR impact
(overhead, flow state)
[25],[27],[37],[38],[39]
[40],[41],[42],[43]
Scenario Full SR
[25],[27],[28],[29],[30]
[31],[32],[33],[34],[36]
[38],[40],[41],[42],[43]
Partially deployed SR [26],[37]
TABLE V:
Classification of the references related to Traffic Engineering.

In the following we provide a brief overview of the references classified as TE related works.

[25] implements a TE algorithm with SR in a SDN infrastructure which builds path with bandwidth guarantees and minimizes at the same time the possibility of rejecting traffic demands. Respect to other solutions, it takes into account the link “criticality” and not only link residual bandwidth. Citing [25]: “Link criticality is based on the concept of the minimal interference routing method”. It allows to minimize the possibility of rejecting requests when the network becomes overloaded. The proposed algorithm not only achieves the goal of balancing the network traffic load, but it also promises to reduce the network costs. Since it is based on SR principles, the proposed solution also considers the extra network overhead caused by the segment labels in the packet headers. The path length has been modeled as a constraint of the heuristic adopting an extra hop limitation in order to save network resources - extra bandwidth used by the segment lists in the packet headers.

Bahnasse et al. [26] propose an SDN based architecture for managing MPLS Traffic Engineering Diffserv aware networks which bases its forwarding on SR-MPLS principles. Architecture has been meant to support also hybrid deployments where SDN equipments coexist with legacy devices guaranteeing same forwarding capabilities. Legacy devices are confined in the core of the network while SDN capabilities have to be supported by the edge devices. In this way, once the controller has calculated the paths meeting the SLA parameters of the flows, programs the network interacting with the edge and setting up the SR paths - over the time the architecture monitors the network and dynamically manages the SR-LSPs in order to ensure that the routing realized by segments does not violate the end-to-end QoS constraints.

Segment Routing and Multicast are combined in [27]; authors propose a routing solution for Multicast based on SR and an heuristic with bandwidth guarantees for Multicast tree calculation which takes into account the load of the links, the number of branching points and the state in the network. The objective is to minimize the number of requests being rejected. In particular, the SDN controller computes an explicit Multicast tree using the aforementioned heuristic and then programs the source node of the tree and its branching points: each time a packet reaches a branching point needs to be duplicated and forwarded on different paths and a modification of the segment lists is performed.

Also [28] deals with SR based TE. In particular, authors design an online energy-efficient Traffic Engineering method. Authors use the SDN controller to selectively switch off and put in sleep mode a subset of links. Then, they dynamically adapt the number of the powered-on links to the traffic load. In this work SR is used to dynamically re-route the traffic. First a least-congested link technique is run to identify the eligible links that can be switched-off. At this point new routes are calculated solving via heuristics the “Multicommodity Flow Problem”. Finally, SDN controller enforces through SR the new paths or IGP forwarding is leveraged if the route corresponds to the shortest path. At this point the links not necessary are turned off.

The flexibility in path selection achieved by Segment Routing is exploited in [29] to propose a new energy efficiency routing strategy. The main focus of the work is to switch off a subset of network links, by properly select alternative paths for the traffic currently steered through the target links. Clearly, the new paths must have enough bandwidth resources to handle the new traffic. In order to better exploit the available capacity, differently from other energy aware routing strategies that work at the level of traffic flows, in [29] the alternative path selection is performed at per-packet level. This allows to define a more accurate traffic splitting strategy, that turns in a more efficient use of the available bandwidth, thus increasing the number of switched off links.

Authors in [30] propose an architecture which combines SDN with SR-based TE. An open source implementation of SR-MPLS is provided together with the realization of a SDN control plane which deals with the calculation of the optimal SR paths in the network. Authors start implementing a basic TE heuristic which solves in approximate way the flow assignment problem. The latter allows to minimize the overall network congestion. This first procedure is also used as admission control for the next phase where the admitted paths are mapped onto SR paths using an heuristic of assignment (which has been described extensively in [71] - Section III-E). Performance evaluation analyzes the distribution of path lengths comparing TE paths with the shortest paths and the distribution of the segment list lengths showing that most of the paths can be implemented using 1 or 2 SIDs. All developed code is open source and available at [90]

A theoretical analysis of the computational complexity of the Traffic Engineering problems in Segment Routing enabled networks is provided in [31]. Two different TE problems are considered: i) the throughput maximization, and the ii) maximum link load minimization. As first the General Segment Routing paradigm is considered. In such a scenario, segments are not constrained to follow shortest paths, but can represents any (possible) complex path. The resolution of both the aforementioned TE problems results to be NP hard. This finding provide a theoretical foundation to the reason why, in Segment Routing, shortest paths are considered for each segment. Then, the complexity of the TE problems is studied for the case of Segment Routing with shortest path. An interesting outcome of this analysis is that, when the number of segments to be used for each segment list is fixed, then the problem of minimizing the maximum link utilization can be solved in weakly polynomial time. Despite this, when the length of the segment lists is only upper bounded (not fixed), then the investigated TE problems fall again in the NP hard class.

In [32]

, the authors deal with SR-based TE designing solutions for the optimal allocation of traffic demands using an ECMP-aware approach. The authors proposes two optimal solutions for online and offline optimization using a 2-segment allocation. The latter consists in the computation for each flow the optimal segment list of two segments with the objective of minimizing the overall network congestion. Key idea of this work is to minimize the worst-case link utilization by considering ECMP forwarding in the offline case. While in the online case, the traffic split values are properly computed also to minimize rejections of requests. Performance evaluation shows that the n-segment routing problem (“Multicommodity Flow Problem”) is just slightly better than 2-segment routing problem but the computation complexity is higher due to more degrees of freedom.

[33] proposes an extension of the models presented in [32]. In particular, the authors propose the 3-segments forwarding demonstrating that the one defined in [32], using two segments, is not sufficient to determine the optimal paths and leads to the wasting of bandwidth.

DEFO (Declarative and Expressive Forwarding Optimizer) is a two layer architecture, described in [34], which is realized on top of a physical communication network, aiming at providing a flexible and highly programmable network infrastructure. At the bottom of the architecture there is a connectivity layer, which is in charge of providing default paths between the network routers. In DEFO, the connectivity layer is represented by an Interior Gateway Protocol (IGP). By means of an optimization layer, the routing paths of a subset of traffic demands is deviated by the default behaviour, provided by the connectivity layer, and is steered through a set of optimized paths. DEFO exploits the flexibility of Segment Routing to implement the optimization layer and configure optimized paths on top of underlying routing paths. The Service Provider can program the network leveraging a high level interface that allows to define specific network goals through the use of a Domain Specific Language (DSL). Once the goal has been specified, DEFO starts the computation of optimized paths, by running an algorithm that exploits the concepts of Middle-point Routing (MR) and Constraint Programming (CP).

[35] faces the problem of fast reacting to sudden traffic changes. In fact, these unexpected events, which occurs at low time scale, can create temporary congestion on links, thus degradating the network performance. Classical solutions to face this issue are based on MILP models or Constraint Programming, see [32] and [34]. Unfortunately, these approaches suffer to high computation time, since they work in a time scale of seconds or minutes, providing TE routing strategies that on average allow to reduce the network congestion, but that might incur in link overloading due to sudden traffic spikes. For this reason, in [35] authors present an algorithm which aims at mitigating link congestion under hard time constraint. The proposed solution exploits SR to fast and flexibly re-route a subset of flows, so that to decrease the maximum link utilization. Time constraint is taken into account under two different perspectives: i) it is directly consider as hard constraint during the algorithm execution, i.e, it is a termination condition, and ii) the selected routing strategy has to be as close as possible to the current one, in order to minimize the number of reconfiguration needed to make it working. These two requirements are simultaneously satisfied by the proposition of a Local Search (LS) based heuristic, which takes as input an initial solution and iteratively goes from that solution to another one, by applying local changes called moves, until a stop criterion is met (e.g. the solution is good enough, or a time limit). Results show that the proposed algorithm overcomes MILP or Constraint Programming based heuristic, allowing for a significant reduction of network congestion with execution times lower than 1 second.

The Segment Routing Path variable is introduced in [36] with the aim of reducing the memory space and the computation time to formulate and solve TE problems in SR networks. In fact, while the memory space needed to instantiate classical TE problem formulations based on links, paths or nodes variables do not scale well with the size of the considered network and with the number of demands to be routed, SR path variables promise to increase efficiency in the problem resolution. SR Path variables are based on the concept of Forwarding Graph (FG). An FG is a Direct Acyclic Graph (DAG) that originates from a node s and terminates on a sink node t. The path followed by a demand in the network is encoded as a sequence of FGs, and this sequence is stored into an SR path variable. Array-based sparse-sets are used to implement SR path variables. A Large Neighborhood Search approach is used to compute optimized paths for the demands. The idea is to iterative improve the best-so-far solution trying to reassign the value of a subset of SR path variables, related to demands that are critical (e.g. all the ones that are currently routed over the most loaded link).

[37] investigates the problem of migrating an IP network in a full SR enabled one. The idea is that, the process of upgrade the system of IP routers in order to enable SR capabilities is done incrementally, so that to reduce the chance of introducing possible misconfigurations or unavailability of the service. To do that, the Segment Routing Domain (SRD) is defined as the subset of SR capable nodes. Depending on the fact that the SRD is a connected set or not, two different models are proposed: Single-SRD or Multiple-SRD. Two main advantages of S-SRD model are that it limits the number of flow states to be maintained at the edges of the SRD, and the average length of the segment list is restrained. As a main drawback there is a potential decrease in the flexibility in the definition of network paths. On the contrary, M-SRD allows to define more complex paths at the cost of having an higher number of flow states and an higher average segment list length. The Segment Routing Domain Design problem is formulated as an ILP, where the main goal is to maximize the Traffic Engineering opportunities, i.e., the identification of the subset of nodes, of a given size, to be upgraded with SR capabilities, so that to have the highest possible flexibility in balancing the links load in the network. The proposed formulation is able to capture both S-SRD and M-SRD models.

[38] proposes ILP models and heuristics for TE applications in SR-based networks. Three ILP models are proposed and they are only used as benchmark for the heuristics due to their high computational complexity. The first is able to leverage ECMP forwarding, the second one computes single routes and implements an hop-by-hop forwarding. Finally the third one is able to leverage the full capability of Segment Routing. An heuristic has been implemented as some instances of the ILP models require too much time to be solved. The heuristic computes an unique route for each flow and tries to keep the total and the maximum network utilization as low as possible. Moreover, it is able to guarantee that the maximum value of the segment list depth is not exceed.

[39]

proposes a TE solution for paths computation that, leveraging at most three labels, is able to optimize link resources and avoid congestion in the network. Firstly, the SDN controller address the problem of properly compute the weights of the IGP Link State protocol using Evolutionary algorithms. Then, the traffic distribution is computed through the Distributed Exponentially-weighted Flow SpliTting (DEFT) or the Penalizing Exponential Flow-spliTting (PEFT) which assign the flows to a next-hop with a probability that decreases exponentially with the extra length of the path (with the respect of the shortest path). SR is used to achieve detours and implement the traffic splitting. Performance evaluation shows that the TE solution delivers a lower congestion with the respect of OSPF/ECMP with optimized configurations and is able to use all available links.

The flexibility of SR in path selection, together with the higher throughput provided by the Multi Path TCP (MPTCP), are exploited in [40] in order to optimize the throughput for large flows and cope with the explosive growth of multimedia traffic, in 5G networks. The propose architecture uses a centralized control plane, with a central controller in charge of managing the Quality of Experience of each MCTCP connection. In particular, the central controller finds multiple paths for each connection, checks the bandwidth requirements and install specific flow rules at the ingress nodes of the considered 5G network. When a new MPTCP connection is established, the controller must find a path for each of the subflows belonging to this connection. For each subflow, it first check the flow path and resource database, in order to check whether there is already a pre-computed path that has enough bandwidth to support the new subflow. In case this check fails, then the controller calculates a new path, encodes it into a segment list and install a new flow entry in the ingress switch. The algorithm used to compute the new paths, named QoE-Centric Multipath Routing Algorithm (QoMRA), tries to find multiple disjoint paths by while considering QoE requirements.

[41] proposes to use Multi Path TCP (MPTCP) and SR to maximize the throughput of the traffic flows in a Data Center network. MTCP allows to split a single connection over several paths, increasing then the bandwidth utilization. Clearly, it also drastically increases the overall number of traffic flows, by making scalability issue to arise. SR is used to reduce the number of flow rules needed to steer the sigle TCP connections over disjoint paths.

In [42], ELEANOR, a northbound application for the OpenDayLight (ODL) Software Defined Network (SDN) controller is presented. ELEANOR considers MPLS-SR data plane, where a Maximum Stack Depth (MSD) constraint, i.e., an upper bound on the number of sids that can be stacked in a segment list, has to be considered. The main components of ELEANOR are: i) a path computation module, and ii) a label stack optimization module. When a new request arrives at the controller, it first finds a suitable path in order to meet specific QoS requirements (bandwidth, delay, etc.), then the appropriate SL is produced.

In [43]

, authors proposes two routing algorithms based on SR for realizing TE applications in SDN networks. These algorithms search for an appropriate selection of the links weights for optimizing paths costs and balancing load across the links. This is obtained through the multiple objective particle swarm optimization (MOPSO) algorithm. Two objective functions are used to measure path cost and load balancing, respectively. According to the authors, their algorithms not only reduce the cost of the paths, better balance the network load, and decrease the maximum link utilization rate but also can improve the satisfaction ratio with respect of shortest path first (SPF), shortest widest path (SWP), widest shortest path (WSP), minimum interference routing algorithm (MIRA), and the Lee algorithm.

Iii-C Failure Recovery

Nine research works dealing with SR for Failure Recovery and Network Resiliency have been published in the last years. The proposed solution can be classified considering the type of failure they are able to recover from:, i.e.link or node failures. Table VI shows the classification of the covered papers.

Node failure [45],[46],[47],[48],[49],[50]
Link failure
[44],[45],[46],[48],[49],[50]
[51],[52]
TABLE VI:
Classification of the references related to Failure Recovery.

In the following we provide a brief overview of the references classified as Failure Recovery related works.

[44] deals with resilient SR forwarding. In particular, authors focus on static fast failover solutions for Segment Routing which do not require the interaction with control plane, the algorithm, the so called TI-MFA, has interesting performance guarantees and it is also resilient to multiple failures while traditional SR fast failover based on TI-LFA can work only with a limited number of failures. Firstly, authors demonstrate that TI-LFA loops indefinitely also for two link failures. Then, a robust but inefficient solutions is shortly presented which pre-compute routing rules considering destination, incident failures and incoming port of the packets. Even if this solution can provide better resiliency guarantees, it introduces some inefficiencies (in terms of path lengths) even if only one link failure occurs. Finally, authors present their solution which basically proposes to store the already hit failures in the packet header and, each time a new failure is faced in the network, the segment list is re-computed using the pre-computed local table entries and the state stored in the packets.

Traffic duplication through disjoint paths is explored in [45]. In particular, authors leverage SRv6 to realize a traffic duplication service which can guarantee an 1+1 protection scheme through the use of disjoint paths. Main difference with other protection mechanisms is that with 1+1 protection both channels are active and data is sent over both paths. Authors use mirroring behavior in the Linux kernel to realize the traffic duplication. Work builds upon the results of [91], in particular it leverages the Linux implementation of SRv6. Then, authors propose an algorithm that is able to compute disjoint paths with least latency and that can be implemented with a number of segments (they set an upper bound limit on their number and use only node segments).

[46] extends [45] with the introduction of the robustly disjoint paths. Authors built, extending routing theory and leveraging configuration synthesis, an automated compiler which is proactive, fast and self-healing (no external intervention are required): it computes pairs of disjoint paths for given sources and destination routers which are robust in the way that they remain disjoint even upon a set of failures. This is achieved without requiring any intervention thank to SR technology. Indeed, SR allows to write sequences of segments which maps to different disjoint paths even when there are topological changes. Finally, leveraging the results of [45], Aubry et al. added to the compiler the capability of limiting the number of segments (i.e. path encoding problem) and computing paths that do not degrade data-plane performance (finding low latency SR paths - addressing also TE aspects).

In [47]

, Hao et al. propose a linear programming model to optimize the restoration in SR based networks. Key idea of the optimized restoration is to share properly the remaining bandwidth when several failures happen, and this is addressed through an optimal configuration of the initial segments, knowing in advance the traffic matrix and the network topology. In particular, authors develop an efficient primal-dual algorithm which can handle single link failures and multiple logical link failures at the same time (including node failures). Moreover, with a simple randomized rounding scheme they can take into account also ECMP forwarding in the network.

A logically centralized implementation of the SR control plane (SDN based) is leveraged in [48], [49] and [50] describe a method to dynamically recover the network from link or node failures. The failover mechanism envisages a failover table for each interface of the node and when a port goes down automatically the related secondary table is used to implement a loop-free backup path from the point of failure to the destination node. Firstly, the node pops all the labels in the segment list except the last label (which represents the final destination), and then the packet processing is passed to the proper failover table. Authors provides an implementation based on the OF protocol leveraging the Group tables feature for monitoring and backup actions.

Same authors of [89] propose an open source implementation of SRv6 TI-LFA using eBPF in [51]. The fast rerouting solution is implemented as a custom BPF program compiled on the fly and attached to a route. In particular a program is loaded into the kernel for each link to be protected. The repair list associated to the route is computed by the control plane and then hard-coded in the eBPF program which is subsequently compiled and installed in the kernel. The solution has been complemented with a robust failure detection architecture which implements through SRv6 the BFD’s echo mode. In particular, the architecture envisages for each link a master node periodically sending probes which can in its turn activate the fast rerouting mechanism described so far. The probes are sent with a special segment list which allows: to redirect the traffic to a special BPF program (BPF slave) on the remote peer and to loop back the probes to the master. The BPF slave, runing in the remote peer, handles the probes packets and can activate in its turn the SRv6 TI-LFA mechanism for its side of the link. The master uses SRv6 type-length-value to insert sequence numbers and timestamps which allow also the remote peer to detect failures. Performance evaluation measures the number of false positives due to an overloaded CPU, BPF implementation of the peer nodes allows to reduce almost to zero the false positives even when the failure detection is less than . Instead the master node is still largely affected by the overloading of the CPU since for sending the probes it has to use an user space process.

[52] study the problem of defining a backup path scheme that is robust to the presence of multiple link failures. Specifically, the main contributions are: i) introduction of a polynomial-time fast rerouting algorithm which allows to define a backup path scheme for resilience under simultaneous link failures in particular case of a hyper cube topology, and ii) the formalization, by means of an ILP formulation, of the problem of defining a backup path scheme that maximizes the number of allowed simultaneous link failures in general graphs.

Iii-D Centrally Controlled Architectures

The definition of Centrally Controlled Architectures exploiting the SR architecture has been widely investigated in literature, resulting in fifteen different works. The solution proposed has been classified on the basis of three different aspects, as reported in Table VII. Considering the SR dataplane, an high number of works makes use of the SR-MPLS dataplane while only three works are based on SRv6. Moreover, few works do no explicitly consider a specific SR implementation but simply exploit the SR possibility of inserting the flow state in the packet header. A further aspect used to classify the research papers is the protocol used for the southbound communication between the network devices and the central controller. The considered protocols are Openflow, PCEP or others. The last component that differentiates the proposed architectures is the underlay network. Specifically, the network devices can be IP/MPLS routers or Openflow switches.

SR Dataplane SR concept [59],[60],[65]
SR-MPLS
[53],[54],[55],[56]
[57],[58],[64],[66]
SRv6 [61],[62],[63]
Southbound
Interface
Openflow [53],[57],[58],[60]
PCEP [54],[55],[56],[64]
Other [63], [64],[66]
Underlay
Dataplane
IP/MPLS
[54],[55],[57]
[58],[64]
IP/SDN
[53],[59],[60],[61]
[62],[63]
TABLE VII:
Classification of the references related to Centrally Controlled Architectures.

In the following we provide a brief overview of the references classified as Centrally Controlled Architectures related works.

[53] implements an SDN based SR-MPLS architecture in a multi-layer packet-optical network. In particular, authors demonstrate optical bypass upon traffic load variations without requiring GMPLS operations. The RYU SDN controller [92] has been extended to control the labels stack configuration at the edge nodes (Open vSwitch based). The SDN controller utilizes OF 1.3 to program the edge devices. Open vSwitch has been modified to increase the maximum MPLS stack depth from 3 to 15 labels. The OF protocol has been modified as well to push all the required labels as a single flow entry and with a single action. Commercial ROADMs have bee used to provide an optical bypass between nodes and provide alternative paths during path computation.

Later in [54], SR has been implemented in a multi-layer network using a PCE architecture instead of SDN/OF. In this scenario nodes consist of commercially available IP/MPLS routers and the SR Controller is an extended version of PCE stateful control plane. Extensions to the PCE protocol allow a centralized PCE to control the label stacking configuration. PCC (PC client - devices) uses the SR-PCE-CAPABILITY type-length-value for specifying the capability of handling SR-enabled Label Switched Paths and the capability to perform SR computation. The Explicit Routing Object (ERO) carried out in the Path Computation Reply message contains the list of computed SIDs and/or the Node or Adjacency Identifier depending on the SID type. On the device the agent collects the information derived from IGP protocol and configures the related shortest path entries and the SID labels. When a new PC replay message is received, the label stack is properly configured. Also this SR implementation using PCE is able to perform dynamic packet rerouting (with optical bypass capabilities), by enforcing different segment-lists at source node, without any signalling protocol and with no packet loss.

In [55] Segment Routing is proposed as enabling technology to realize Network Service Chaining (NSC) in a metro-core network scenario, where service chain requests are represented by the so called micro flows, i.e., a huge number of low or medium bit-rate flows. In this scenario, classical solutions based on MPLS or pure SDN fails due to scalability issues. On the contrary, SR moves the flow state into the packet header, reducing the configuration costs (and time) to the installation of the encapsulation rule at the ingress point of the network (the one used to add the segment list to each packet of the considered flow). Based on this consideration, [55] describes an SR Path Computation Element (SR-PCE), which is in charge of orchestrating connection setup/release/modification, and is made by two main modules: i) the flow computation element, having the goal of find a path with available resources to serve a micro flow and ii) flow steering API that is responsible of installing the SR encapsulation rule in the ingress node. An experiment evaluation of the proposed architecture is proposed in [56].

[57] proposes a SR-based Software Defined Network (SDN) architecture which is able to perform load balancing among ECMP and non-ECMP routes in multi-layer networks including an IP/MPLS layer over an optical network layer. In particular, two solutions are described: i) Centralized-SR; ii) Preconfigured-SR. The former leverages a SDN controller to steer traffic over alternative paths upon network failures. Instead, the second solution uses OpenFlow load-balance groups to actively forward the traffic on several routes. With the second solution the data plane layer can autonomously react to a network impairment removing the failed output port while the first solution always requires the intervention of the SDN controller but results to be more generic with regards of the second one. Both solutions push the SID of the destination node and leverage the available ECMP paths. The recovery property of the architecture has been validated simulating network failures. According to the authors some packet losses have been recorded and the recovery time was around .

The SPRING-OPEN project [58] is an ONOS [93] use case, which provides an open source SDN-based implementation of SR. Its architecture is based on a logically centralized control plane, built on top of ONOS, and it drastically eliminates the IP/MPLS Control Plane from the network. Part of this work converged later in the Trellis project [94], an SDN based leaf-spine fabric, built using bare-metal hardware, open-source software from the OCP [95] and ONOS projects, and an open-API from Broadcom [96] to program merchant-silicon ASICs (OF-DPA) [97]. The leaf-spine fabric is based on SR-MPLS principles, however it does not implement the full fledged SR architecture and it just uses global significant Node-SIDs. These MPLS labels are statically configured in the SDN control plane and are used to globally identify the ToR switches of the fabric and routes the traffic towards them using a single MPLS label.

In order to provide resiliency against link and node failures for Cloud Service Provides (CSPs), an overlay infrastructure realized by means of Segment Routing and Software Defined Networking control plane is proposed in [59]. The main idea of the proposed architecture is to substitute the dedicated physical infrastructure that interconnects Data Centers of a CSP, with an overlay network realized on top of an underlay infrastructure, represented by the interconnection of many Internet Service Provider (ISP) networks. The prerequisite is the availability of multi-homed connections for each Data Center of the CSP. Then, the logical component of the proposed overlay infrastructure are: i) a central controller that monitors the underlay network status and determines new paths in case of a failure, ii) the egress points that are responsible of routing the traffic flows leaving a Data Center toward the most appropriate ISP, and iii) the routing inflection points that are special nodes that manage the routing between two distant Data Centers, by using SR encapsulation.

[60] proposes the use of SR in an hybrid IP/SDN network as a technique to mitigate the problem of the limited storage space in the flow tables of the SDN switches. The main goal is to use SR to optimize the use of the flow tables and the link capacity. In the considered scenario, every node supports both OpenFlow operation and normal IP switching operation. When a packet enters a node, first it is classified in order to decide through which pipeline it has to be steered, then it is processed accordingly. Operations associated to the normal IP pipeline are decided using a routing protocol (eg. OSPF). Differently, the path to be followed by traffic flows steered through the OpenFlow Switching Layer (OFSL) are decided by the central SDN controller. In order to limit the number of flow entries to install in order to configure a path, SR is exploited. In this way, a portion of the flow state information is moved from the flow table of the switches to the packet header. In order to insert the SR related information, i.e., the segment list, in the packet header, [60] proposes to use unused fields (eg. VLAN tag or optional fields).

Software Resolved Networks (SRNs) is a new SDN architecture recently proposed for IPv6 enterprise networks [61][62]; further details about the implementation are provided in the Section V. The network is managed by a logically centralized controller which interacts with the end-hosts through an extended DNS protocol: applications are allowed to embed traffic and/or path requirements in their requests and the controller is able to return the appropriate path to the applications satisfying their needs. SRv6 is used as dataplane technology to steer the traffic on a specific path according to the network policies. Each component that can be reached through the network is always referenced through a DNS name. The default DNS resolver in the hosts is modified to interact with the controller of the architecture. SRN provides also a mechanism for the dynamic registration of the end-points. In this way, the DNS database can be properly updated and the name resolution can be performed by the clients. The connections are always unidirectional so it is necessary to establish two paths in order to enable the communication between the endpoints. Binding segment is used to implement a path id and it is automatically translated in a SRv6 policy in the access node. Path segmentation is performed using the algorithm illustrated in [19] which allows to match a given policy and guarantee the minimal list of segments. In order to optimize the interaction with the controller, upon a request the controller computes the two paths to support the communication and then instruct the access device of a node to add also the reverse binding segment in the SRH. In this way, the reply can be simply echoed back. Software Resolved Network has been implemented on Linux end-hosts, routers and controllers.

In [63] a novel SDN architecture is proposed for SRv6 technology; Section V provides further information about the implementation and where is possible to download the code. The dataplane is constituted by Linux based SRv6 nodes built from open source components which expose an open API towards the SDN controller. The nodes result to be hybrid as they envisage the coexistence of a legacy IP control plane with an SDN control plane. Authors present the design and implementation of the Southbound API between the SDN controller and the SRv6 devices which is used to instantiate SRv6 policies in the network. In particular they propose a data-model and provide four different implementations of the API, respectively based on gRPC, REST, NETCONF and remote Command Line Interface (CLI). Topology discovery is also addressed actively extracting the topology database from the IP routing daemons running in the network nodes.

A hierarchical multi-domain control plane for SDN networks based on SR has been demonstrated in [64]. The control plane is composed by an orchestrator application which runs on top of multiple SDN controllers and leverages their NB APIs to create multi-domain SR based services. BGP-LS and PCEP are used as southbound in the SDN controllers and provide respectively network topology and the creation of MPLS SR tunnels. IS-IS is used inside the domain to exchange reachability information and SIDs between nodes. SDN controllers do not exchange any reachability information nor SIDs. Orchestrator interacts with the SDN controllers and builds a global network view that will be used to perform the path computation and to instantiate SR services. A practical demonstration has been realized using software routers.

In [65] two solutions for multi-domain SR are proposed: end-to-end Segment Routing and per-domain Segment Routing. Both methods leverage a non-standard east/west interface between peer controllers, thus rely on a flat control-plane architecture and do not use signaling sessions in the data plane. In the first approach, the segment list contains already the end-to-end path crossing several domains. The originator domain sends a request to the destination domain. The destination controller computes the segment list to reach the destination from its ingress router and sends it back to the previous domain. Each intermediate domain applies the same procedure stitching the segment list computed by the downstream until the reply is received back by the originator. Conversely in the second approach, the end-to-end path is obtained stitching several SR paths: in each domain the segment list contains a virtual label as bottom of the stack which triggers a modification of the segment list in the ingress border node of the next domain. In this case, there is no global view of the network, and controllers do not know the domain sequence to reach destination.

[66] proposes an advance of Carrier Ethernet architecture and envisages an approach mixing SR and SDN technologies. It results to be a trade-off between fully-distributed control planes and centralized approaches: an inventory database is maintained into the SDN controller with the configuration for each device. The SDN controller provides the IP configuration and SR configuration like the loopback address, node label, label range, gateway label information via a southbound API like NETCONF/Yang. For any communication inside the domain by design the network will use the IGP based forwarding without the need of the SDN controller and will impose on the traffic a single MPLS label (loopback SID). Multiple labels can be used to realize TE applications. Instead, in a multi-domain scenario several SDN controllers are deployed and exchange reachability information in order to properly program the edge nodes. In this way an inter-domain path can be established simply with a label stack that includes local and remote border router labels, plus the end node label.

Iii-E Path Encoding

The translation of a network path, resulting from a specific TE objective, into a sequence of SIDs, i.e. a Segment List, is a key operation for the deployment of SR in a real network. This operation is usually referred to as Path Encoding. Seven different papers have been defined for the definition of a proper a solution to the path encoding problem. The different algorithms differs depends on the nature of the network path, i.e. the path to be encoded. Specifically, the input path might be computed on top of a network with uniform link weight or arbitrary ones. Moreover, it might include ECMP or not. A further aspect for the path encoding algorithms is the possibility of requiring additional device configurations, as the insertion of a new policy. Finally, path encoding algorithms can differ depending on the possibility of encoding the input path with one or more SLs. Table VIII shows the paper classification according to the aforementioned aspects.

Path to be
encoded
uniform
IGP weights
[67],[68],[69]
arbitrary
IGP weights
[70],[71],[72],[73],[74]
ECMP
not allowed
[67],[69],[70],[71]
ECMP
allowed
[68],[72],[73],[74]
Requires
further configuration
no
[67],[68],[69]
[70],[71],[72]
yes [73],[74]
Encoded path single SL
[67],[68],[69],[70]
[71],[73],[74]
multiple SLs [72]
TABLE VIII:
Classification of the references related to Path Encoding.

In the following we provide a brief overview of the references classified as Path Encoding related works.

In [67], SDN and PCE based implementations of SR controller share a common path engine, that performs the hop-by-hop path computation and SR path assignment. As regards the path computation engine, the controller selects the least congested path on a set of candidate paths. Then, the proposed SR path assignment algorithm provides the shortest Segment list considering a unique path towards the destination and avoiding load balancing through ECMP. The algorithm uses two pointers and to navigate the target path from source to destination. Firstly, is incremented until the considered sub-path is no more a unique shortest path. At this point, the SID of the node is inserted in the Segments list and the two pointers are both set to and the procedure restarts. This cycle is repeated until the node is equal to the destination. The algorithms provides the segment list of minimum depth but the solution only considers global Node-SID therefore it cannot be applied to topologies with arbitrary IGP link costs.

Authors of [68] propose a Segment list encoding algorithm to express a given path, which minimizes (enforcing a give threshold) the Segment list depth in SR-based networks. It considers ECMP forwarding by default, but can also introduce constraints to support a deterministic hop-by-hop path. Respect to other efforts, the solution is not able to support arbitrary hop-by-hop paths when arbitrary IGP link costs are used. The algorithm is based on graph computation. The so called auxiliary graph is built firstly using the physical links of the paths which are computed by an external TE heuristic, subsequently virtual links are added representing the ECMP paths between two nodes. Each virtual link is annotated with the metrics and the number of the ECMPs between the two nodes. Using this auxiliary graph a new path computation is performed adding a new constraint related to the number of hops: each path having a number of hop greater than the maximum Segment list depth is rejected. Then the candidate paths are sorted first according to their original metrics, second according to their length and then according to the number of ECMPs. Finally the paths are translated in SIDs using this approach: physical links are changed with their respective Adjacency SID, virtual links are mapped with the Node SID of the destination node. If the Adjacency SID are not local the algorithm first insert the Node SID of the source and then the Adjacency SID.

[69] proposes two algorithms for the computation of segment list which result to be optimal in terms of stack depth when an unique hop-by-hop path has been computed. Key idea is to consider at each iteration bigger sub-paths and verify if an unique shortest path exists, substitute the sub-path with the tailnode SID and then move to the remaining part of the path. The algorithms are very similar, main difference is how to vary the breadth of the sub-paths until considering the original hop-by-hop path. The first algorithm navigates the target path starting from the source node toward the destination, while the second one leverages the opposite direction. The analysis of the overhead in the packet headers shows that reverse algorithm introduces less overhead compared to the direct algorithm, as the computed segment list typically includes nodes that are near the source.

Also in [70], two algorithms for an efficient paths encoding, the so called SR-LEAs, are proposed. The algorithms take as input an explicit shortest path and then compute the relative segment list having as constraint a given maximum segment list depth. They are composed by two main steps: i) computation of successive shortest paths; ii) labels replacing. Specifically, they compute the subpaths of the original shortest path and take into account the limitation of the hardware to reduce the number of the subpaths. In the second step, the subpaths composed of three or more nodes are replaced by the Node-SID of their tailnode. SR-LEA replaces two nodes subpaths using the Adj-SID. The variant SR-LEA-A is very similar to the above algorithm but takes advantage of the global Adj-SID to further reduce the depth of the labels stack. Of course, it requires the advertisement of these SIDs to properly work. Simulation results show that the algorithms are able to compute segment lists with an average length lower than 3. SR-LEA-A delivers best results and can improve the segment list computation compared to SR-LEA but the gain is around 5% in terms of average length.

Authors of [71] propose an optimal SR path assignment algorithm and prove that it is optimal in terms of number of used segments. The algorithm takes as an input an hop by hop path computed by a TE heuristics and computes an SR path congruent with the one calculated by the heuristic and composed by a minimum number of segments. The SR assignment algorithm evaluates at each iteration if it exists a shortest path between the current source and the current destination, and then each time considers different subpaths updating the current source or the current destination. Each time a new segment is found the current source is updated with the previous destination and the current destination is substituted by the destination of the original hop-by-hop path. In the worst case, the link between the current source and its next hop is evaluated, if the link is different from the shortest path an adjacency segment is pushed in the segment list.

The authors of [72] propose a two-step method for translating a given Traffic Engineering (TE) path into an SL. In the first step, an auxiliary graph is created; the aim of the auxiliary graph is to represent all the Interior Gateway Protocol (IGP) paths available, i.e. forwarding paths, for the specific TE path to be encoded. In the second step, a MILP problem over the auxiliary graph is defined: the MILP problem allows to minimize the overall Segment List length given the TE path to implement. One of the main feature of the propose solution is the multi-path support, i.e. the ability to split a source-destination flow among a weighted set of segment lists.

In [73] the problem of optimizing the performance of an SR network under the maximum Segment List Depth (max-SLD) constraint is studied. In fact, especially when SR is realized on top of the MPLS data plane, the constraint on the max-SLD is particularly limiting. A possible solution is to create new LSPs, so that to increase the availability of alternative paths in the underlay network, and consequently being able to write shorter segment lists. Despite the creation of new LSPs allows to reduce the length of the segment lists, it has as main drawbacks an increase on the number of required forwarding rules to install in the forwarding tables of the routers. In order to mitigate this negative effect, [73] defines the concept of panel based forwarding: a panel refers to a set of node-disjointed LSPs that can be represented by the same label. Furthermore, an ILP formulation is proposed to solve the path encoding problem, while minimizing both the number of new defined LSPs and install rules, while respecting the max-SLD constraint.

To overcome the Maximum Stack Depth (MSD) constraint in SR-MPLS, in [74] a new type of SID, named Targeted SID (TSID), is defined. A TSID is a local segment that is associated with a sequence of SIDs. The instruction related to a TSID consists in replacing it in the SL with the associated sequence of SIDs. By using a TSID is then possible to reduce the length of a SL, at the cost of introducing a new flow state in the node that implements the instruction related to the TSID. For this reason, [74] proposes two different optimization problems that allows to find a trade-off between the benefits and the costs of the implementation of the TSIDs. The first optimization problems takes as input the set of paths whose related SL overcomes the MSD bound, and aims at minimizing the number of defined TSIDs. The focus is then the reduction of the number of extra flow states to be maintained by the network nodes. A second optimization problem is presented, and has the goal of minimizing the PCEP sessions that has to be maintained between the central controller, that is responsible for the TSID installation, and the nodes where the TSIDs have to be installed. In this case, the main idea is to install in the same node as many TSIDs as possible.

Iii-F Network Programming

Despite the fact that Network Programming is the most attractive and innovative feature of SR, we have found only eight papers related to this topic. As shown in table IX, these works can be classified in Service Function Chaining related ones, or operational function. The latter aim at implementing operational functions, such as a load balancer and a zero-loss VM migration tool, by exploiting the network programming feature of SR.

Service Function
Chaining
[75],[76],[77],[78],[82]
Operational Function [79],[80],[81]
TABLE IX:
Classification of the references related to Network Programming.

In the following we provide a brief overview of the references classified as Network Programming related works.

In [75] the Linux SRv6 implementation described in [91] is enhanced introducing the support for Service Function Chaining. More in detail, the Linux kernel provides an API to map a service segment, i.e. the identifier of a network service running on a Virtual Machine. The experimental evaluation show that the impact of SR operations and service segment processing on the packet forwarding capabilities is limited, i.e. with a reduction lower than .

The architecture of a network domain supporting Service Function Chaining (SFC) through SRv6 is investigated in [76]. The authors mainly focus on the implementation of a VNF node able to host multiple VNF instances. The main components of the VNF nodes are the SR/VNF connector, in charge of logically connecting the SR routing with local VNFs, and the VNFs, supporting a specific network functions. The VNFs can be SR-aware or SR-unaware: i) SR-aware VNFs can process the information contained in the SR header (SRH) of incoming packets; ii) SR-unaware VNFs are not able to handle the SRH thus, in order to correctly apply the VNF to the original packet, the SR/VNF connector must pre-process the packet by removing the SR encapsulation, and re-apply it when the packet is returned by the VNF. The authors propose a Linux-based implementation of a VNF node supporting both SR-aware and SR-unaware VNFs. More in detail, using the netfilter framework, a new kernel module called srext (Segment Routing EXTensions) is implemented to act as a SR/VNF connector and support SR-unaware VNFs. A virtualized testbed based on Virtualbox ang Vagrant is realized to evaluate the processing overhead introduced by the proposed implementation with respect to a classical IPv6 forwarding solution.

SRV6Pipes [77] is an extension of the IPv6 implementation of Segment Routing in the Linux kernel which enables chaining and operation of in-network functions operating on streams. SRv6 is used to enforce an end-to-end path between the client and the server passing through the equipment hosting the networking functions. The rationale behind SRv6Pipes stands in the fact that some network functions need to include a TCP implementation to work on the streams. SRv6Pipes leverages “the TCP stack that is already present in the Linux kernel” and implements a transparent TCP proxy to offload to it the TCP functionalities and terminates the TCP connections where a network function is deployed. In this way, it is possible to expose to the network functions the bytestreams they need to process. Special addressing is used to specify network functions and their parameters. In particular each proxy exposes an 80 prefix. The first 80 bits are used to traverse the proxy, the following 16 bits are used to identify the network function and the remaining ones are used to specify the parameters of the function.

[78] defines the concept of SR Aware VNF, as an application that is able to process the SR information in the packet. Moreover, [78] also proposes the implementation an SR Aware (SERA) Firewall application. The SERA Firewall is able to work both as a legacy firewall (basic mode), or define filtering rules that also include condition on the SR fields (advanced mode). In the basic mode, it can apply the normal firewall processing to the original packets even if they have an SR based SFC encapsulation. It means that the filtering rules are applied to the original values of the packet header (the SR related fields are transparent). In the advanced mode, SERA offers new matching capabilities and new SR specific actions. Some examples are the seg6-go-next, which sends the packet toward the next segment in the SL, the seg6-skip-next which allows to skip the next SID in the SL, or the seg6-go-last which sends the packet directly to the last SID of the SL.

Segment Routing Load Balancing (SRLB) [79] is an Application-aware load-balancer that avoids the cost due to the monitoring tasks. It is thought to work in the contest of a Data Center network, where several instances of the same application are instantiated in different hosts machines. Each host machine is equipped with a VPP based virtual router which dispatches packets between physical NICs and application-bound virtual interfaces. The Load Balancer (LB) is located at the edge of the Data Center network. A request for an application is represented by the first packet sent from the client to the application server (generally a syn message). When a new request arrives at the LB, the Service Hunting function, which consists in finding a server that can serve the current request, is executed. LB exploits SR to query a subset of servers that host the requested application. Specifically, LB encodes the set of randomly selected potential servers in the segment list, then encapsulates the first packet of the request. When the first server receives the packet, it can decide whether to deliver it to the application, and consequently assign to it the processing of this request, or can forward the packet to the next server in the segment list. The decision about accept/refuse a request is taken according to a connection acceptance policy, that takes into account the internal state of the application (CPU usage, memory usage, etc.).

[80] proposes a new architecture for the Content Delivery Networks (CDNs) building upon the results of [79]

. In this new paradigm, CDN decisions (cache vs origin servers) are offloaded to the dataplane. This architecture founds on two fundamental pieces: i) chunk-level content addressing and ii) in-network server selection. The first is realized assigning a unique and globally routable IPv6 address to each chunk. Instead, the in-network server selection leverages these identifiers exposed as IP addresses to make in-band forwarding decisions which are later binded to a SRv6 steering policy. In particular, upon arrival of a request at a CDN proxy, the IPv6 identifier is used by the prediction engine to perform cache admission by estimating the popularity of requests with a Least-Recently-Used (LRU) filter. If not available (cache miss), 6LB

[79] is used to forward requests directly to the origin servers instead of proxying them at the cache. According to the authors, this mechanism allows to reduce the load on the edge cache and avoid the negative effects on the Quality of Experience.

[81] proposes an SR based live migration technique which achieves zero packet loss. The starting point is an SRv6 enabled Data center network. Two new SR functions are defined and used in the process of VM live migration. The first one is forward to local if present, which is a conditional version of the END function. Specifically, in case the last SID in the SL is locally available, then the packet is directly processed, thus ignoring possible intermediate SIDs. The second one is buffer and forward to local, that forces the node to inspect the last SID and, in case it is not locally available, the packet is buffered until the SID becomes available. Now, assuming that a VM is migrated from host A to host B, then during the migration process all the requests received by the Data center gateway are tunneled using an SL where: i) the first SID points at the forward to local if present function implemented at A, ii) the second SID points at the buffer and forward to local function implemented at B, and iii) the last SID is referred to the VM. In this way, until the VM is available at A, the packets are directly delivered to the VM thanks to the forward to local if present function. Then, during the downtime, the packets directed to the VM are buffered at B thanks to the buffer and forward to local. Finally, when the VM becomes available at B, the buffered packets, as well as new arrivals, are directly delivered to the VM at B. In this way it is possible to implement VM live migration having no packet loss.

SRNK [82] is a SR-proxy for legacy VNFs which are not aware of the SRv6 technology and expect to process traditional IP packets. SRNK extends the implementation of SRv6 in the Linux kernel [98] adding the support for the End.AS and End.AD behaviors. The performance of the proposed solution has been evaluated by the authors which identified a poor scalability with respect to the number of VNFs to be supported within an NFV node. With an enhancement of the Linux Policy Routing framework, they provided a second design SRNKv2, which does not depend on the number of supported VNFs in a node. They compared the performance with a reference scenario not performing the encapsulation and decapsulation operation and demonstrated that the overhead of SRNKv2 is very small, on the order of 3.5%.

Iii-G Miscellaneous

SRPerf [83] is a performance evaluation framework for software and hardware implementations of SRv6. SRPerf is able to perform different benchmarking tests such as throughput and latency. At the time of writing, the framework supports only Linux kernel implementation of SRv6. The architecture of SRPerf can be easily extended to support new benchmarking methodologies as well as different SRv6 implementations. The framework supports two different metrics to characterize the throughput of a SRv6 enabled node: No-Drop Rate (NDR) and Partial Drop Rate (PDR). PDR is defined as the highest throughput achieved without dropping packets more than a predefined threshold; NDR can be described as PDR with threshold of 0%. The framework orchestrates all the aspects of an experiment starting from the setup of the testbed up to the enforcement of the SRv6 configurations, relieving the experimenter from a significant configuration effort. SRPerf has been used to evaluate the performance of the SRv6 implementation in the Linux kernel.

Even if Control Exchange Point [84] does not explicitly leverage SR, it proposes the concept of pathlets [85] which closely resembles to the idea of the list of instructions present in the SR architecture. Control Exchange Point (CXP) main goal is to provide services with QoS constraints across domains. This is achieved stitching the pathlets which are partial paths advertised by the domains. An ISP abstracts its network as a set of pathlets connecting the network edges and then advertises these on the northbound. More specifically, this abstraction is realized with tunnels instantiated with OF, MPLS, optical path and so on. The pathlet abstraction is bundled with properties that the ISP provides like latency, costs, available bandwidth and so on. A CXP is an external entity acting as brokering layer and providing inter-domain routing coordination based on SDN APIs. The general idea seems to be inline with SR architecture presented so far and it can be implemented using SR data-plane technologies.

[86], [87] and [88] propose an alternative implementation of SR architecture through Omnipresent Ethernet (OE), which is a modification of Carrier Ethernet architecture. It is based on source-routed, binary-routed labels embedded in an Ethernet frame. [88] provides details on the implemented SR header.

Iv Standardization activities

In this section we propose a classification and description of the standardization activity related to Segment Routing. The number of standardization documents related to Segment Routing is high, thus we decided to focus on a subset of them: more in detail, we report here 9 Request For Comment (RFC) and 50 Internet Drafts.

Architecture [5, 99, 100, 101, 6, 8, 102]
Use-case and Requirements [2, 3, 4, 103, 104, 105, 106, 107, 108, 109, 110, 111]
Fast Reroute (FRR) [112, 113, 114]
OAM [115, 116, 117, 118, 119]
Performance Measurement [120, 121, 122, 24]
Protocol Extensions Data Plane SR-MPLS [123, 124, 125, 126, 127, 128, 129, 130, 131, 132]
SRv6 [7, 133, 134]
Control Plane BGP [135, 136, 137, 138, 139, 140, 141, 142, 143]
BGP-LS [144, 145, 146, 147, 148, 149, 150, 151, 152, 153]
IS-IS [14, 154, 15, 155, 156, 157, 158, 159, 160, 161, 162]
OSPF
[12, 163, 13, 164]
[165, 166, 16, 167, 157, 168]
PCEP [17, 169, 170]
LISP [171]
TABLE X:
Classification of the Standardization Activities documents

We first propose a Taxonomy to classify each standardization document, and then in the next subsection we report an overview of the key standardization activities . The result of this classification is shown in Table X. The first category is Architecture, where all the documents regarding the description of the general architecture of a Segment Routing network are considered. The RFC 8402 falls into this category and describes the main features of SR, such as the source routing paradigm idea, the concept of SID and the definition of the two supported data planes. In the category Use-case and Requirements the documents describing use case scenarios for SR, e.g., use of SR in WANs, data center networks, mobility and network slicing, are inserted. Specifically, in this category there are 3 RFCs: i) RFC 7855 introducing the Source Packet Routing in Networking (SPRING), ii) RFC 8355 related to network resiliency using SR, and iii) RFC 8354 that describes how to steer IPv6 or MPLS packets over the SPRING architecture. The third category is FRR one, i.e, Fast Reroute realized through SR. The main standardization activity in this category is related to fast recovery after a link failure, and is referred to as Topology Independent Loop Free Alternate (TI-LFA), described in [112]. No RFC has been published in this category. Operations, Administration, and Maintenance (OAM) is the fourth defined category, where we include all the standardization activities related to tools used for maintenance of the network. As example, RFC 8287 focus on the implementation of the ping and traceroute tools in SR. In the Performance Measurement category we consider all the documents describing measurement procedures related to performance parameters, such as delay and packet loss, in an SR network. We include in this category also the two RFCs 6374 and 7876 that explain how to measure delay and packet loss in MPLS. Despite these two documents have not been produced during the standardization activities of SR, we decided to include them in Table X since they are massively used in the drafts for performance monitoring regarding SR. Finally, the Protocol Extensions category covers two different set of documents related to extensions of legacy protocols: i) data plane protocols extensions, and ii) control plane protocols extensions. At the data plane, we consider two drafts describing SR-MPLS [131] and SRv6 [7]. At the control plane, we focus on documents on modifications to routing protocol (eg. BGP and OSPF) for the dynamic distribution of the SIDs in the SR network, or control protocol for the communication between a central controller (in case of centralized control plane) and the devices at the data plane (eg. PCEP). In the next subsections we provide an overview of the selected documents of the standardization activity.

Iv-a Key standardization efforts

In this subsection, we provide an overview of the most important standardization efforts. [5] and [99] define key tenets of the SR architecture and discuss the benefits brought by SR in terms of scalability, privacy and security. [102], [106] and [107] elaborate more on the support of key use cases like NFV/SFC, SD-WAN and next generation of mobile networks. Instead, [8] extends basic SR concepts and [112] provides Fast Re-route (FRR) mechanisms against single failures. Finally, [113] and [157] analyzes the improvements of the routing stability and extensions to the routing protocols.

[5] describes the Segment Routing architecture and its overall design. It defines the concept of a segment as a network instruction and presents the basic types of segments: prefix-SID, adjacency-SID, peering-SID and binding-SID. It also explains how such segments can be attached to data packets, leveraging the MPLS or IPv6 data planes, in order to steer traffic flows on any path in the network without requiring any per-flow state in the fabric.

[99] details the concept of an SR policy. It explains how Candidate Paths are defined as explicit SID-lists or as dynamically computed paths based on some optimization criteria, and how the active Candidate Path is selected. Moreover, it presents various ways of steering traffic into an SR Policy, automatically by coloring BGP service routes, remotely using a Binding-SID, or statically with route policies. The concepts described in this draft equally apply to the MPLS and SRv6 dataplanes.

The SR architecture is extended from the simple steering of packets across nodes to a general network programming approach in [8]. Using this framework, it is possible to encode arbitrary instructions and not only locations in a SID-list. Each SID is associated with a function to be executed at a specific location in the network. A set of basic functions are defined in [8], but other functions can be defined by network operators to fit their particular needs. Moreover, SID arguments allow functions to be provided additional context or their behavior to be tweaked on a per-flow basis.

[102] defines the service SIDs and describes how to implement service programming in SR-MPLS and SRv6 enabled networks. Key tenet is to associate to a SID to each network function (either physical or virtual). These service SIDs may be combined together in a SID-list and finely programmed by leveraging the network programming concept. They can also be combined with other types of SIDs to provide traffic engineering or VPN services. Service segments can be associated to legacy appliances as well (with no SR capabilities), thanks to SR proxy mechanisms which perform the SR processing and hides the SR information from the network function.

In [106] it is explained how SR technology enables underlay Service Level Agreements (SLA) to a VPN with scale and security while ensuring service opacity. SR based VPNs are analyzed in the case of a single provider and multiple providers. Moreover, the draft addresses also the control plane aspects of such solution which are managed by an over the top SD-WAN controller. Finally, the benefits brought by the SR technology to VPN services are analyzed in term of scalability, privacy, billing management and security.

[107] describes the applicability of SRv6 to the user plane of mobile networks. Three modes are addressed: traditional mode, enhanced mode and enhanced mode with interworking. In the first, mobile user plane is unchanged except for the use of SRv6 as transport instead of GTP-U. Enhanced mode uses only SRv6 and its programming framework. Finally, the last mode uses SRv6 but provides also interworking functionality with legacy components still using GTP. Moreover, the document defines the SID functions for the SRv6 mobile user plane and describes a mechanism for end-to-end network slicing.

Topology Independent Loop-free Alternate (TI-LFA) [112] provides Fast Re-route (FRR) mechanisms protecting against link, node or local Shared Risk Link Groups (SRLGs)failures in SR enabled networks. For each destination in the network, a backup path is pre-computed and installed in the forwarding table, so that it is ready to be activated as soon as a failure is detected. For each destination, the backup path matches the post-convergence path, which is followed by the traffic after the network convergence. The draft analyzes also the benefits of using Segment Routing with the respect of traditional FRR solutions.

[113] describes a mechanism leveraging SR to prevent transient routing inconsistencies during the convergence period that follows a network topology modification. Instead of directly converging to a new next-hop after a topology modification, a node can temporarily steer the impacted traffic through a set of loop-free SR Policies, thus preventing it from being affected by routing inconsistencies. After the network has fully converged, the temporary SR Policies are removed with no impact on the traffic.

[157] defines a set of extensions to the IGP routing protocols that enable Prefix-SIDs to be associated with operator-defined shortest path algorithms, called SR Flexible Algorithms (Flex-Algo). These algorithms are defined as an optimization metric (IGP, TE or delay) and a set of constraints (e.g., resources to be excluded from the path). Each node participating in a Flex-Algo computes the shortest paths to the Prefix-SIDs of that Flex-Algo and installs them in it forwarding table. SR Flexible Algorithms allow traffic to be steered along traffic-engineered paths such as low-latency or dual-plane disjoint path with a single Prefix-SID.

V SR implementations and deployments

In this section, we describe the implementation results related to SR. We will mostly focus on the SRv6 version which is attracting a lot of interest and development efforts. The SR-MPLS version is already in a mature development status, well supported by the main core router vendors (e.g. Cisco, Huawei, Juniper). SR-MPLS can be incrementally deployed in current IP-MPLS backbones, as it only requires software updates to networking devices. Operators can migrate to SR-MPLS to simplify the control plane operations and improve the scalability. As for the SRv6 dataplane, there are two main Open Source dataplane implementations for software routers: the Linux kernel implementation (described in Subsection V-A) and the realization done inside the FD.io VPP project (described in Subsection V-B). Section V-C elaborates more on the implementation results related to the research activities. Finally, in Subsection V-D we briefly analyze the hardware implementations of SR, the inter-operability efforts done by several vendors and the currently planned deployments of SRv6 in large production networks.

V-a Linux kernel

The SRv6 capabilities were first added in Linux kernel 4.10 [98]. Kernel 4.10 includes the support for some SRv6 transit behaviors (e.g., T.Insert and T.Encaps). The transit behaviors are implemented as Linux Lightweight Tunnel (LWT). The implementation of the iproute2 [172] user space utility is extended to support adding a localsid associated with an SRv6 transit behavior [173]. SRv6 localsids with transit behavior are added as IPv6 FIB entries into the kernel main routing table. Kernel 4.14 is another important milestone for the SRv6 support in Linux: a set of SRv6 endpoint behaviors have been implemented by adding a new type of LWT [174]. The supported SRv6 endpoint behaviors are End.X, End.T, End.DX2, End.DX4. End.DX6, End.DT6, End.B6, and End.B6.Encaps. Some new transit behaviors have been added (e.g., T.Encaps.L2). The iproute2 implementation was extended as well [172] [175]. The SRv6 capabilities in Linux kernel were extended in kernel 4.16 [176] to include the netfilter framework [177]. A new iptables match extension, named srh, was added to the kernel to support matching of SRH fields. The srh match extension is a part of the SERA firewall [178] and supports matching all the fields of the SRH. The implementation of iptables user space utility [179] is extended with a new shared library (libip6t_srh) that allows to define iptables rules with srh options. Kernel 4.18 [180] has seen some more features both in the core SRv6 stack and the netfilter framework.

In the netfilter framework, the srh match is extended to provide the matching of SRH’s Previous SID, Next SID, and Last SID. The iptables user space utility is updated as well to support the new matching options. Instead, a new feature is added in the Linux SRv6 stack to support custom SRv6 network functions implemented as small eBPF [181] programs. [89] extends [8] introducing a new End behavior the so called End.BPF. From an implementation point of view a new hook for BPF is added to the SRv6 infrastructure that can be used by network operators to attach small programs written in C to SRv6 SIDs which have direct access to the Ethernet frames. Moreover, specific SRv6-BPF helpers have been provided in order to allow End.BPF functions to execute basic SRv6 actions (End.X, End.T and many others) or adding TLVs. This allows also to implement custom SRv6 transit behaviors (mainly to extend SRv6 encapsulation policies implemented by the kernel). The tutorial about eBPF extensions to SRv6 is available at [182]. The source code of the sample applications described in [89] is freely available at [183]. Instead, the eBPF-based fast-reroute and failure detection schemes described in [51] is available at [184].

SR-MPLS has not received the same attention of the SRv6 implementation in the Linux kernel. All the features which are available are mostly related to the well-established MPLS forwarding. They have been made available from the version 4.1 of the kernel. In particular, kernel v4.1 has seen the introduction of the MPLS Label Switching Router (LSR) behavior. MPLS capabilities have been extended later in the kernel v4.3. LWT framework and MPLS tunnel were added allowing the implementation of the MPLS Label Edge Router (LER) behavior. Finally, MPLS multipath functionality has been added only in the version 4.5 of the kernel.

In general, the Linux kernel lacks of the support of the SR policy framework which is instead available for FD.io VPP implementation (Subsection V-B). This means that at the time of writing is not possible to create a SR policy (both MPLS and IPv6) and associate a BindingSID to it nor instantiate SR-MPLS/SRv6 steering rules pointing to SR-MPLS/SRv6 policies.

V-B FD.io VPP

FD.io VPP [185] 17.04 included the support for the transit behaviors and most of the endpoint behaviors defined in [8]. These behaviors are implemented in dedicated VPP graph nodes. The SRv6 graph nodes perform the required SRv6 behaviors as well the IPv6 processing (e.g. decrements Hop Limit). Whenever an SRv6 segment is instantiated, a new IPv6 FIB entry is created for the segment address pointing to the corresponding VPP graph node. Release 17.04 also brought SR headend capabilities to VPP by introducing the concept of SR policy in the SRv6 implementation. In VPP, an SR policy is uniquely identified by its BindingSID address, which serves as a key to a particular SR policy. This is not compliant with the SR policy definition [99], but a reasonable shortcut considering the absence of control-plane capabilities in VPP.

The SR policies in VPP support several SID lists with weighted load-balancing of the traffic among them. When a new segment list is specified for an SR policy, VPP pre-computes the rewrite string that will be used upon steering traffic into that SID list, either via a transit behavior or a BindingSID. VPP then initializes one FIB entry for the SR policy BindingSID in the FIB and an entry in a hidden FIB table for the IPv6 traffic steered into the SR policy via a transit behavior. Each one of these FIB entries points to the SR policy object, which in turn recurses on the weighted segment lists.

Traffic can be steered into an SR policy either by sending it to the corresponding BindingSID or by configuring a rule, called steering policy, that directs all traffic transiting towards a particular IP prefix or L2 interface into a SRv6 policy. The latter mechanism is implemented as FIB entry for the steered traffic in the main FIB to be resolved via the FIB entry of the SR policy in the hidden FIB table. In this way, a hierarchical FIB structure is realized: the traffic is not directly steered over an SR policy, but instead directed to a hidden FIB entry associated with the policy. This allows the SR policy to be modified without requiring any change to the steering rules that point towards it.

Release 17.04 has also seen the introduction of the SRv6 LocalSID development framework and the SR-MPLS implementation. The former is an API which allows developers to create new SRv6 endpoint behaviors using the VPP plugin framework. The principle is that the developer only codes the actual behavior, i.e. the VPP graph node. Instead, the segment instantiation, listing and removal are performed by the existing SRv6 code. The SR-MPLS framework is focused on the SR policies, as well on its steering. Likewise in SRv6, an SR policy is defined by a MPLS label representing the BindingSID and a weighted set of MPLS stacks of labels. Spray policies are a specific type of SR-MPLS policies where the packet is replicated on all the SID lists, rather than load-balanced among them. Tto steer packets in transit into an SR-MPLS policy, the user has to to create an SR-MPLS steering policy. Instead, others SR-MPLS features, such as for example adjacency SIDs, can be achieved using the regular VPP MPLS implementation. In release 18.04, service programming proxy behaviors End.AS, End.AD and End.AM were introduced as VPP plugins leveraging the framework described before.

V-C Rest of us

Most of the research efforts analyzed in this work have released as open source the components and the extensions realized for SR. Some of the them build upon the implementation described in the previous Subsections, while other propose alternative solutions. The SREXT module ([76]) provides a complementary implementation of SRv6 in Linux based nodes. When it was designed, the Linux kernel only offered the basic SRv6 processing (End behavior). SREXT complemented the SRv6 Linux kernel implementation providing a set of behaviors that were not supported yet. Currently most of the behaviors implemented in SREXT are supported by the mainline of Linux kernel (with the exception of the SR proxy behaviors). SREXT provides an additional local SID table which coexists with the one maintained by the Linux kernel. The SREXT module registers itself as a callback function in the pre-routing hook of the netfilter [177] framework. Since its position is at beginning of the netfilter processing, it is invoked for each received IPv6 packet. If the destination IPv6 address matches an entry in the local SID table, the associated behavior is applied otherwise the packet will follow the normal processing of the routing subsystem. The source code of SREXT together with the Vagrant box are available at [186] which allow to bootstrap a small testbed in few minutes and experiment with SREXT features.

FRRouting (FRR) [187] is an open source routing protocol stack for Linux forked from Quagga [188]. In FRR, there is an experimental support [189] of the draft [12] which defines the OSPFv2 extensions for Segment Routing (SR-MPLS). At the time of writing, there is no support for SRv6.

The SPRING-OPEN project [58] provides an SDN-based implementation of SR-MPLS. The architecture is based on a logically centralized control plane, built on top of ONOS. Part of this work converged later in the Trellis project [94], an SDN based leaf-spine fabric which has been designed to be multi-purpose and multi-vendor. Trellis supports also P4/P4Runtime [190] [191] devices as well as Stratum [192] enabled devices. Trellis has been used as underlay/overlay fabric in the CORD project [193] which aims at redesigning central-office architectures. Recently, it has been integrated in the SEBA project [194] which targets residential-access networks. All the software stack and the documentation is freely available on [94]. Moreover, a tutorial together with a ready-to-go VM can be downloaded from [195].

PMSR ([71] and [30]) provides an open source implementation of SR-MPLS together with the realization of a SDN control plane. The data plane leverages the OSHI architecture ([196], [197]) which combines a SDN data plane, implemented with Open vSwitch [198], and OSPFv2 control logic, realized with Quagga. This architecture is extended in PMSR with the introduction of a Routes Extraction entity which connects to Quagga and receives routes update using the FPM interface provided by Quagga [188]. These routes are then translated in SIDs and installed in the SDN data plane as OpenFlow MPLS forwarding rules. Authors provide a set of management tools [199] which assist experimenters and relieve them from a huge configuration effort. A tutorial to start working with PMSR is available on [200]; instead a ready-to-go VM with all the dependencies installed can be downloaded from [201].

Software Resolved Network (SRN) (described in [61] and [62]) is a variant of the SDN architecture. The network controller is logically centralized and co-located with a DNS resolver and uses extensions of the DNS protocol to interact with end-hosts. OVSDB [202] is used to enable the communication between SDN controller and the network nodes: i) the latter populates the distributed database with the topology information and TE metadata; ii) the former once computed the path, upon a request, populates the OVSDB instance with the SRv6 Segment list matching the desired requirements. Finally, this is pulled by the access device which enables the communication of the end-hosts. The source code is freely available at [203]. An overview of the architecture can be found in [204]. A ready-to-go VM with packaged experiments can be created using the instructions in [205].

[63] proposes a classical SDN architecture for SRv6 technology: a centralized logic takes decisions on the Segment Lists that need to be applied to implement the services, then the SDN controller, using a southbound API, interacts with the SR enabled devices to enforce the application of such Segment Lists. The code related to the SDN architecture, i.e. the four different implementations of the Southbound API and the topology discovery, can be downloaded from the page of the project [206]. In addition the authors, to support both the development and testing aspects, have realized an Intent based emulation system to build realistic and reproducible experiments relieving the experimenters from a huge configuration effort. The emulation tools are available at [207].

SRV6Pipes [77] is an extension of the SRv6 implementation in the Linux kernel [98] which enables chaining and operation of in-network functions operating on streams. SRv6 policies are installed using the SRN architecture [61] described earlier in this section. However, SRN components are not mandatory since SRv6 policies can be installed in the edge nodes using the iproute utility. SRv6Pipes is composed by multiple components that necessarily need to be installed in the target machines: TCP proxy, patched Kernel and user space utilities. The minimum components can be downloaded from the repository of the project [208]. The complete code of the experiments together with a walkthrough can be found in [209].

SRNK [82] extends the implementation of SRv6 in the Linux kernel [98] adding the support for the End.AS and End.AD behaviors. The source code is freely available at [210], where it is possible to download the patched Linux kernel (starting from 4.14.0 branch) and the patched iproute2 (starting from iproute2-ss171112 tag). Instead, the detailed configurations steps of the SR-proxy are reported in the Appendices A and B of [82].

[46] describes the implementation of a path computation element able to compute robust disjoints SR paths which remains disjoint even after an input set of failures without the need of configuration changes. The java implementation of the algorithms, the public topologies used for the experiments, the experimental results and a detailed walkthrough to replicate the experiments of the paper are available at [211].

V-D Hardware implementations, inter-operability efforts and planned deployments

[212] provides an overview of IPv6 Segment Routing implementations and details some interoperability scenarios that have been demonstrated at public events. As regards the hardware implementations, the platforms ASR 1000, ASR 9000, NCS 5500, NCS 540 and NCS560 are reported as the Cisco Routing platforms supporting SRH processing [213]. The programmable devices based on Tofino chipset [214] can be programmed to support SRH processing. This is also true for the reference software implementation of the P4 devices [215], Stratum based devices [192] and all the programmable chipset (Cavium Xpliant [216] to give an example). The draft elaborates also on the open source applications supporting the processing of the IPv6 Segment Routing header, among which we mention the well known Wireshark [217], tcpdump [218], iptables [219], nftables [220] and snort [221].

The implementations, described in the previous paragraph, have been used in interoperability testing scenarios showcased at the 2017 SIGCOMM conference [222]. The set of experiments included a L3 VPN scenario augmented with TE functionality and services function chaining processing. SREXT, VPP, Linux kernel, Barefoot Tofino, Cisco NCS5500 and Cisco ASR1000 routers were the network devices implementing SRv6 behaviors. While iptables and snort have been used as service functions. Finally, Wireshark and tcpdump have been leveraged to verify the proper operations of the network.

[7] reports the implementation status of other vendors. Juniper’s Trio and vTrio NPUs has an experimental support of SRH (SRH insertion mode and End processing of interfaces addresses). Instead, Huawei’s VRP platform is in production stage and has the capability of adding SRH header and performing End processing.

To conclude the section, we mention that large scale deployments of SRv6 in production networks have been recently announced. In particular two mobile operators are planning SRv6 deployments in their “pre-5G” or “5G-ready’ networks ([223, 224]).

Vi Conclusion

The Segment Routing technology is based on the source routing and tunneling paradigms. Segment Routing supports services like Traffic Engineering, Virtual Private Networks, Fast Restoration in IP backbones and datacenters and proved to be flexible in supporting new use cases. Moreover, the SR architecture reduces the amount of state information that needs to be configured in the core nodes.

In the survey we investigated research works and standardization activity related to the Segment Routing paradigm. We also provided a detailed evaluation of real implementations, focusing on available tools to realize experimental testbed for SR research and development activities.

SR-MPLS and SRv6 are the two dataplane instantiations of the SR architecture. This is the first tutorial and survey work covering in detail the novel implementation of SR over IPv6, i.e. SRv6, that represents the most promising implementation for future research activity. SRv6 provides a consistent solution for solving long-lived problems in the IP networks, simplifying protocol stacks and improving scalability with respect to current solutions.

We covered more than 75 scientific papers related to SR and we proposed a taxonomy for their classification. One of the main outcome of the classification was to identify relationships between the SR features and the research topics. For instance, the source routing paradigm has turned to be the key enabling feature for the implementation of Traffic Engineering solutions in an SR network, while the routing flexibility feature is mainly used to realize network monitoring tools. We also identified the most interesting SR standardization documents, providing a taxonomy for their classification. The study on the SR implementations highlighted the maturity of the solutions based on Linux kernel and VPP. Linux and VPP, widely used by the research and developer community, allow to easily deploy a virtual SR playground.

To conclude, we try to provide guidelines for future directions of the SR research activities. To begin with, we believe that more emphasis should be given to the Programmability features of SR, that can be fruitfully exploited in the contexts of NFV and SFC. In this regard, new abstraction models for the control of Segment Routing network could be studied. Another interesting the topic is related to moving SR functions from the end-host stacks closer to the hardware. SmartNICs allow to implement network traffic processing on the NIC instead of using the CPU of the end-nodes. There is also a growing interest in the integration of the SRv6 technology into Cloud orchestrators like OpenStack [225] and Kubernetes [226]. The goal is to replace the current dataplane mechanism based on legacy tunneling mechanisms like VXLAN with a drastic simplification of the network stack. Moreover, the use of SR in the 5G networks and IoT is a promising research topic, above all in conjunction with SRv6. Finally, there are a lot of efforts in supporting field deployments as witnessed in several recent announcements [227].

We hope that this tutorial and survey work will raise the attention of the research community on the Segment Routing technology and motivate new researchers to join the development of new use cases and the standardization efforts. We have anticipated new challenges and future topics earlier in this section which can be picked as starting points. Moreover, we strongly encourage the community to provide feedback and updates as new research works and SR deployments come out and technology evolves. For this reason we have created a public repository 111https://github.com/netgroup/sr-survey, where interested people can contribute and update this documentation.

Acknowledgment

This work has received funding from the Cisco University Research Program Fund.

References

  • [1] S. Yasukawa et al., “An Analysis of Scaling Issues in MPLS-TE Core Networks,” IETF RFC 5439, Feb. 2009. [Online]. Available: https://tools.ietf.org/html/rfc5439
  • [2] S. Previdi et al., “Source Packet Routing in Networking (SPRING) Problem Statement and Requirements,” Internet Requests for Comments, RFC Editor, RFC 7855, May 2016. [Online]. Available: https://tools.ietf.org/html/rfc7855
  • [3] C. Filsfils et al., “Resiliency Use Cases in Source Packet Routing in Networking (SPRING) Networks,” IETF RFC 8355, Mar. 2018. [Online]. Available: https://tools.ietf.org/html/rfc8355
  • [4] J. Brzozowski et al., “Use Cases for IPv6 Source Packet Routing in Networking (SPRING),” IETF RFC 8354, Mar. 2018. [Online]. Available: https://tools.ietf.org/html/rfc8354
  • [5] S. Previdi et al., “Segment Routing Architecture,” IETF RFC 8402, Jul. 2018. [Online]. Available: https://tools.ietf.org/html/rfc8402
  • [6] A. Bashandy et al., “Segment Routing with MPLS data plane,” Internet Engineering Task Force, Internet-Draft draft-ietf-spring-segment-routing-mpls-19, Mar. 2019, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-spring-segment-routing-mpls-19
  • [7] C. Filsfils et al., “IPv6 Segment Routing Header (SRH),” IETF, Internet-Draft, October 2018. [Online]. Available: https://tools.ietf.org/html/draft-ietf-6man-segment-routing-header
  • [8] C. Filsfils et al., “SRv6 Network Programming,” Internet Engineering Task Force, Internet-Draft draft-filsfils-spring-srv6-network-programming-07, Feb. 2019, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-filsfils-spring-srv6-network-programming-07
  • [9] Z. N. Abdullah, I. Ahmad and I. Hussain, “Segment Routing in Software Defined Networks: A Survey,” IEEE Communications Surveys & Tutorials, 2018.
  • [10] C. Filsfils et al., “The Segment Routing Architecture,” Global Communications Conference (GLOBECOM), 2015 IEEE, pp. 1–6, 2015.
  • [11] A. Farrel, R. Bonica, “Segment Routing: Cutting Through the Hype and Finding the IETF’s Innovative Nugget of Gold,” IETF Journal, vol. 13, no. 1, July 2017. [Online]. Available: https://www.ietfjournal.org/segment-routing-cutting-through-the-hype-and-finding-the-ietfs-innovative-nugget-of-gold/
  • [12] P. Psenak et al., “OSPF Extensions for Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-ietf-ospf-segment-routing-extensions-27, Dec. 2018, work in Progress. [Online]. Available: https://tools.ietf.org/html/draft-ietf-ospf-segment-routing-extensions
  • [13] P. Psenak et al., “OSPFv3 Extensions for Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-ietf-ospf-ospfv3-segment-routing-extensions-21, Dec. 2018, work in Progress. [Online]. Available: https://tools.ietf.org/html/draft-ietf-ospf-ospfv3-segment-routing-extension
  • [14] S. Previdi et al., “IS-IS Extensions for Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-ietf-isis-segment-routing-extensions-22, Dec. 2018, work in Progress. [Online]. Available: https://tools.ietf.org/html/draft-ietf-isis-segment-routing-extensions
  • [15] P. Psenak (ed.) et al., “IS-IS Extensions to Support Routing over IPv6 Dataplane,” Internet Engineering Task Force, Internet-Draft draft-bashandy-isis-srv6-extensions-03, Oct. 2018, work in Progress. [Online]. Available: https://tools.ietf.org/html/draft-bashandy-isis-srv6-extensions
  • [16] Z. Li et al., “OSPFv3 Extensions for SRv6,” Internet Engineering Task Force, Internet-Draft draft-li-ospf-ospfv3-srv6-extensions-03, Sep. 2018, work in Progress. [Online]. Available: https://tools.ietf.org/html/draft-li-ospf-ospfv3-srv6-extensions
  • [17] S. Sivabalan et al., “PCEP Extensions for Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-ietf-pce-segment-routing-14, Oct. 2018, work in Progress. [Online]. Available: https://tools.ietf.org/html/draft-ietf-pce-segment-routing
  • [18] F. Paolucci et al., “Interoperable multi-domain delay-aware provisioning using Segment Routing monitoring and BGP-LS advertisement,” in ECOC 2016; 42nd European Conference on Optical Communication; Proceedings of.   VDE, 2016, pp. 1–3.
  • [19] F. Aubry et al., “SCMon: Leveraging Segment Routing to improve network monitoring,” in Computer Communications, IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on.   IEEE, 2016, pp. 1–9.
  • [20] X. Li and K. L. Yeung, “Bandwidth-Efficient Network Monitoring Algorithms Based on Segment Routing,” Computer Networks, 2018.
  • [21] M. Polverini et al., “Routing Perturbation for Traffic Matrix Evaluation in a Segment Routing Network,” IEEE Transactions on Network and Service Management, 2018.
  • [22] A. Cianfrani et al., “A Heuristic Approach to Assess the Traffic Matrix of an ISP Exploiting Segment Routing Flexibility,” in 2018 30th International Teletraffic Congress (ITC 30), vol. 1.   IEEE, 2018, pp. 194–199.
  • [23] M. Polverini et al., “Interface Counters in Segment Routing v6: a powerful instrument for Traffic Matrix Assessment,” Network of the Future (NOF), 2018 9th International Conference on th, 2018.
  • [24] R. Gandhi (ed.) et al., “UDP Path for In-band Performance Measurement for Segment Routing Networks,” Internet Engineering Task Force, Internet-Draft draft-gandhi-spring-udp-pm-01, Jun. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-gandhi-spring-udp-pm-01
  • [25] M. Lee et al., “An efficient routing algorithm based on Segment Routing in Software-Defined Networking,” Computer Networks, vol. 103, pp. 44–55, 2016.
  • [26] A. Bahnasse et al., “Novel SDN architecture for smart MPLS Traffic Engineering-DiffServ Aware management,” Future Generation Computer Systems, 2018.
  • [27] J. Sheu et al., “A scalable and bandwidth-efficient multicast algorithm based on Segment Routing in Software-Defined Networking,” in Communications (ICC), 2017 IEEE International Conference on.   IEEE, 2017, pp. 1–6.
  • [28] R. Camplsrpa et al., “Segment Routing based traffic engineering for energy efficient backbone networks,” in Advanced Networks and Telecommuncations Systems (ANTS), 2014 IEEE International Conference on.   IEEE, 2014, pp. 1–6.
  • [29] K. Ghuman et al., “Per-packet based energy aware segment routing approach for Data Center Networks with SDN,” in Telecommunications (ICT), 2017 24th International Conference on.   IEEE, 2017, pp. 1–6.
  • [30] L. Davoli et al., “Traffic engineering with Segment Routing: SDN-based architectural design and open source implementation,” in Software Defined Networks (EWSDN), 2015 Fourth European Workshop on.   IEEE, 2015, pp. 111–112.
  • [31] G. Trimponias et al., “On Traffic Engineering with Segment Routing in SDN based WANs,” arXiv preprint arXiv:1703.05907, 2017.
  • [32] R. Bhatia et al., “Optimized network traffic engineering using Segment Routing,” in Computer Communications (INFOCOM), 2015 IEEE Conference on.   ImplsEEE, 2015, pp. 657–665.
  • [33] H. Roomi and S. Khorsandi, “Semi-Oblivious Segment Routing with Bounded Traffic Fluctuations,” in Electrical Engineering (ICEE), Iranian Conference on.   IEEE, 2018, pp. 1670–1675.
  • [34] R. Hartert et al., “A declarative and expressive approach to control forwarding paths in carrier-grade networks,” in ACM SIGCOMM Computer Communication Review, vol. 45.   ACM, 2015, pp. 15–28.
  • [35] S. Gay et al., “Expect the unexpected: Sub-second optimization for Segment Routing,” in INFOCOM 2017-IEEE Conference on Computer Communications, IEEE.   IEEE, 2017, pp. 1–9.
  • [36] R. Hartert et al., “Solving Segment Routing problems with hybrid constraint programming techniques,” in International Conference on Principles and Practice of Constraint Programming.   Springer, 2015, pp. 592–608.
  • [37] A. Cianfrani et al., “Incremental Deployment of Segment Routing Into an ISP Network: a Traffic Engineering Perspective,” IEEE/ACM Transactions on Networking, vol. 25, no. 5, pp. 3146–3160, 2017.
  • [38] E. Moreno et al., “Traffic engineering in Segment Routing networks,” Computer Networks, vol. 114, pp. 23–31, 2017.
  • [39] V. Pereira et al.

    , “Optimizing Segment Routing using Evolutionary Computation,”

    Procedia Computer Science, vol. 110, pp. 312–319, 2017.
  • [40] A. A. Barakabitze et al., “A Novel QoE-Centric SDN-based Multipath Routing Approach for Multimedia Services over 5G Networks.”
  • [41] J. Pang et al., “SDN-based data center networking with collaboration of multipath TCP and segment routing,” IEEE Access, vol. 5, pp. 9764–9773, 2017.
  • [42] O. Dugeon et al., “Demonstration of Segment Routing with SDN based label stack optimization,” in Innovations in Clouds, Internet and Networks (ICIN), 2017 20th Conference on.   IEEE, 2017, pp. 143–145.
  • [43] X. Hou, M. Wu, and M. Zhao, “An Optimization Routing Algorithm Based on Segment Routing in Software-Defined Networks,” Sensors, vol. 19, no. 1, p. 49, 2019.
  • [44] K. Foerster et al., “TI-MFA: keep calm and reroute segments fast,” in IEEE INFOCOM 2018-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).   IEEE, 2018, pp. 415–420.
  • [45] F. Aubry et al., “Traffic duplication through segmentable disjoint paths,” in IFIP Networking Conference (IFIP Networking), 2015.   IEEE, 2015, pp. 1–9.
  • [46] A. François et al., “Robustly disjoint paths with segment routing,” in Proceedings of the 14th International Conference on emerging Networking EXperiments and Technologies.   ACM, 2018, pp. 204–216.
  • [47] F. Hao et al., “Optimizing restoration with Segment Routing,” in Computer Communications, IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on.   IEEE, 2016, pp. 1–9.
  • [48] A. Giorgetti et al., “Segment Routing for effective recovery and multi-domain traffic engineering,” Journal of Optical Communications and Networking, vol. 9, no. 2, pp. A223–A232, 2017.
  • [49] A. Giorgetti et al., “Demonstration of dynamic restoration in Segment Routing multi-layer SDN networks,” in Optical Fiber Communication Conference.   Optical Society of America, 2016, pp. Th4G–4.
  • [50] A. Giorgetti et al., “Reliable Segment Routing,” in Reliable Networks Design and Modeling (RNDM), 2015 7th International Workshop on.   IEEE, 2015, pp. 181–185.
  • [51] M. Xhonneux and O. Bonaventure, “Flexible failure detection and fast reroute using eBPF and SRv6,” arXiv preprint arXiv:1810.10260, 2018.
  • [52] K.-T. Foerster, M. Parham, and S. Schmid, “Local Fast Segment Rerouting on Hypercubes,” 2018. [Online]. Available: http://eprints.cs.univie.ac.at/5837/11/2018-hypercubes.pdf
  • [53] A. Sgambelluri et al., “First demonstration of SDN-based Segment Routing in multi-layer networks,” in Optical Fiber Communications Conference and Exhibition (OFC), 2015.   IEEE, 2015, pp. 1–3.
  • [54] A. Sgambelluri et al., “SDN and PCE implementations for Segment Routing,” in Networks and Optical Communications-(NOC), 2015 20th European Conference on.   IEEE, 2015, pp. 1–4.
  • [55] F. Paolucci, “Network service chaining using segment routing in multi-layer networks,” IEEE/OSA Journal of Optical Communications and Networking, vol. 10, no. 6, pp. 582–592, 2018.
  • [56] F. Paolucci et al., “Service chaining in multi-layer networks using Segment Routing and extended BGP FlowSpec,” in Optical Fiber Communication Conference.   Optical Society of America, 2017, pp. W4J–3.
  • [57] P. Castoldi et al., “Segment Routing in multi-layer networks,” in Transparent Optical Networks (ICTON), 2017 19th International Conference on.   IEEE, 2017, pp. 1–4.
  • [58] “Spring Open Project,” available online at https://wiki.onosproject.org/display/ONOS10/Segment+Routing.
  • [59] A. Fressancourt and M. Gagnaire, “A SDN-based network architecture for cloud resiliency,” in Consumer Communications and Networking Conference (CCNC), 2015 12th Annual IEEE.   IEEE, 2015, pp. 479–484.
  • [60] Z. Li et al., “Segment Routing in Hybrid Software-Defined Networking,” in Communication Software and Networks (ICCSN), 2017 IEEE 9th International Conference on.   IEEE, 2017, pp. 160–165.
  • [61] D. Lebrun et al., “Software resolved networks: Rethinking enterprise networks with IPv6 Segment Routing,” in Proceedings of the Symposium on SDN Research.   ACM, 2018, p. 6.
  • [62] F. Duchene et al., “Exploring various use cases for IPv6 Segment Routing,” in Proceedings of the ACM SIGCOMM 2018 Conference on Posters and Demos.   ACM, 2018, pp. 129–131.
  • [63] P. Ventre et al., “SDN Architecture and Southbound APIs for IPv6 Segment Routing Enabled Wide Area Networks,” IEEE Transactions on Network and Service Management, 2018.
  • [64] N. Kukreja et al., “Demonstration of SDN-based orchestration for multi-domain Segment Routing networks,” in Transparent Optical Networks (ICTON), 2016 18th International Conference on.   IEEE, 2016, pp. 1–4.
  • [65] A. Sgambelluri et al., “Experimental demonstration of multi-domain Segment Routing,” in Optical Communication (ECOC), 2015 European Conference on.   IEEE, 2015, pp. 1–3.
  • [66] D. Cai et al., “Evolve carrier Ethernet architecture with SDN and Segment Routing,” in World of Wireless, Mobile and Multimedia Networks (WoWMoM), 2014 IEEE 15th International Symposium on a.   IEEE, 2014, pp. 1–6.
  • [67] A. Sgambelluri et al., “Experimental demonstration of Segment Routing,” Journal of Lightwave Technology, vol. 34, no. 1, pp. 205–212, 2016.
  • [68] F. Lazzeri et al., “Efficient label encoding in Segment-Routing enabled optical networks,” in Optical Network Design and Modeling (ONDM), 2015 International Conference on.   IEEE, 2015, pp. 34–38.
  • [69] A. Giorgetti et al., “Path encoding in Segment Routing,” in Global Communications Conference (GLOBECOM), 2015 IEEE.   IEEE, 2015, pp. 1–6.
  • [70] R. Guedrez et al., “Label encoding algorithm for MPLS Segment Routing,” in Network Computing and Applications (NCA), 2016 IEEE 15th International Symposium on.   IEEE, 2016, pp. 113–117.
  • [71] S. Salsano et al., “PMSR-Poor Man’s Segment Routing, a minimalistic approach to Segment Routing and a Traffic Engineering use case,” in Network Operations and Management Symposium (NOMS), 2016 IEEE/IFIP.   IEEE, 2016, pp. 598–604.
  • [72] A. Cianfrani et al., “Translating traffic engineering outcome into Segment Routing paths: The encoding problem,” in Computer Communications Workshops (INFOCOM WKSHPS), 2016 IEEE Conference on.   IEEE, 2016, pp. 245–250.
  • [73] H. Liaoruo et al., “Optimizing Segment Routing with the Maximum SLD Constraint using Openflow,” IEEE Access, 2018.
  • [74] R. Guedrez et al., “A new method for encoding MPLS Segment Routing TE paths,” in Network of the Future (NOF), 2017 8th International Conference on th.   IEEE, 2017, pp. 58–65.
  • [75] D. Lebrun, “Leveraging IPv6 Segment Routing for Service Function Chaining,” in CoNEXT 2015 student workshop, 2015.
  • [76] A. Abdelsalam et al., “Implementation of virtual network function chaining through segment routing in a linux-based NFV infrastructure,” in 2017 IEEE Conference on Network Softwarization (NetSoft), July 2017, pp. 1–5.
  • [77] F. Duchêne et al., “Srv6pipes: enabling in-network bytestream functions,” in IFIP Networking 2018, 2018.
  • [78] A. Abdelsalam et al., “SERA: SEgment Routing Aware Firewall for Service Function Chaining scenarios,” in IFIP Networking, 2018.
  • [79] Y. Desmouceaux et al., “SRLB: The Power of Choices in Load Balancing with Segment Routing,” in Distributed Computing Systems (ICDCS), 2017 IEEE 37th International Conference on.   IEEE, 2017, pp. 2011–2016.
  • [80] Y. Desmouceaux et al. A Content-aware Data-plane for Efficient and Scalable Video Delivery.
  • [81] Y. Desmouceaux et al., “Zero-Loss Virtual Machine Migration with IPv6 Segment Routing,” in 1st Workshop on Segment Routing and Service Function Chaining (SR+SFC 2018) at CNSM 2018, Rome, Italy, 2018.
  • [82] A. Mayer et al., “An Efficient Linux Kernel Implementation of Service Function Chaining for legacy VNFs based on IPv6 Segment Routing,” arXiv preprint arXiv:1901.00936, 2019.
  • [83] A. Abdelsalam et al., “Performance of IPv6 Segment Routing in Linux Kernel,” in 1st Workshop on Segment Routing and Service Function Chaining (SR+SFC 2018) at CNSM 2018, Rome, Italy, 2018.
  • [84] V. Kotronis et al., “Control eXchange Points: Providing Qos-enabled end-to-end services via SDN-based inter-domain routing orchestration,” LINX, vol. 2429, no. 1093, p. 2443, 2014.
  • [85] P. Godfrey et al., “Pathlet routing,” ACM SIGCOMM Computer Communication Review, vol. 39, no. 4, pp. 111–122, 2009.
  • [86] S. Bidkar et al., “Scalable Segment Routing: a new paradigm for efficient service provider networking using carrier Ethernet advances,” IEEE/OSA Journal of Optical Communications and Networking, vol. 7, no. 5, pp. 445–460, 2015.
  • [87] S. Bidkar et al., “A scalable framework for Segment Routing in service provider networks: The omnipresent Ethernet approach,” in High Performance Switching and Routing (HPSR), 2014 IEEE 15th International Conference on.   IEEE, 2014, pp. 76–83.
  • [88] S. Bidkar et al., “Field trial of a Software Defined Network (SDN) using carrier Ethernet and Segment Routing in a tier-1 provider,” in Global Communications Conference (GLOBECOM), 2014 IEEE.   IEEE, 2014, pp. 2166–2172.
  • [89] M. Xhonneux et al., “Leveraging eBPF for programmable network functions with IPv6 Segment Routing,” arXiv preprint arXiv:1810.10247, 2018.
  • [90] “SDN-TE-SR,” available online at https://github.com/netgroup/SDN-TE-SR.
  • [91] D. Lebrun, “A Linux kernel implementation of Segment Routing with IPv6,” in Computer Communications Workshops (INFOCOM WKSHPS), 2016 IEEE Conference on.   IEEE, 2016, pp. 1019–1020.
  • [92] “RYU Project,” available online at http://osrg.github.io/ryu/.
  • [93] Open Networking Foundation, “ONOS Project,” available online at https://onosproject.org.
  • [94] “Trellis,” available online at https://wiki.opencord.org/display/ CORD/Underlay+Fabric.
  • [95] “Open Compute Project,” available online at http://www.opencompute.org.
  • [96] “Broadcom,” available online at https://www.broadcom.com.
  • [97] Broadcom, “OpenFlow Datapath Abstraction,” available online at https://github.com/Broadcom-Switch/of-dpa.
  • [98] D. Lebrun and O. Bonaventure, “Implementing IPv6 Segment Routing in the Linux Kernel,” in Proceedings of the Applied Networking Research Workshop.   ACM, 2017, pp. 35–41.
  • [99] C. Filsfils et al., “Segment Routing Policy Architecture,” Internet Engineering Task Force, Internet-Draft draft-ietf-spring-segment-routing-policy-02, Oct. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-ietf-spring-segment-routing-policy-02
  • [100] C. Filsfils et al., “SR Policy Implementation and Deployment Considerations,” Internet Engineering Task Force, Internet-Draft draft-filsfils-spring-sr-policy-considerations-02, Oct. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-filsfils-spring-sr-policy-considerations-02
  • [101] J. Thomas et al., “YANG Data Model for Segment Routing Policy,” Internet Engineering Task Force, Internet-Draft draft-thomas-spring-sr-policy-yang-00, Jul. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-thomas-spring-sr-policy-yang-00
  • [102] F. Clad et al., “Service Programming with Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-xuclad-spring-sr-service-programming-01, Oct. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-xuclad-spring-sr-service-programming-01
  • [103] C. Filsfils et al., “BGP-Prefix Segment in large-scale data centers,” Internet Engineering Task Force, Internet-Draft draft-ietf-spring-segment-routing-msdc-11, Nov. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-ietf-spring-segment-routing-msdc-11
  • [104] C. Filsfils et al., “Segment Routing Centralized BGP Peer Engineering,” Internet Engineering Task Force, Internet-Draft draft-ietf-spring-segment-routing-central-epe-10, Dec. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-ietf-spring-segment-routing-central-epe-10
  • [105] C. Filsfils et al., “Interconnecting Millions Of Endpoints With Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-filsfils-spring-large-scale-interconnect, Mar. 2019, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-filsfils-spring-large-scale-interconnect
  • [106] D. Dukes et al., “SR for SDWAN - VPN with Underlay SLA,” Internet Engineering Task Force, Internet-Draft draft-dukes-spring-sr-for-sdwan-01, Dec. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-dukes-spring-sr-for-sdwan-01
  • [107] S. Matsushima et al., “Segment Routing IPv6 for Mobile User Plane,” Internet Engineering Task Force, Internet-Draft draft-ietf-dmm-srv6-mobile-uplane-04, Mar. 2019, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-ietf-dmm-srv6-mobile-uplane-04
  • [108] P. Camarillo et al., “Segment Routing IPv6 for mobile user-plane PoCs,” Internet Engineering Task Force, Internet-Draft draft-camarillo-dmm-srv6-mobile-pocs-01, Oct. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-camarillo-dmm-srv6-mobile-pocs-01
  • [109] Z. Ali et al., “Building blocks for Slicing in Segment Routing Network,” Internet Engineering Task Force, Internet-Draft draft-ali-spring-network-slicing-building-blocks-00, Jul. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-ali-spring-network-slicing-building-blocks-00
  • [110] C. Filsfils, Z. Ali (ed.) et al., “Segment Routing Traffic Accounting Counters,” Internet Engineering Task Force, Internet-Draft draft-filsfils-spring-sr-traffic-counters-00, Jul. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-filsfils-spring-sr-traffic-counters-00
  • [111] M. Anand, S. Bardhan et al., “Packet-Optical Integration in Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-anand-spring-poi-sr-05, Feb. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-anand-spring-poi-sr-05
  • [112] A. Bashandy et al., “Topology Independent Fast Reroute using Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-bashandy-rtgwg-segment-routing-ti-lfa-04, Mar. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-bashandy-rtgwg-segment-routing-ti-lfa-04
  • [113] A. Bashandy et al., “Loop avoidance using Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-bashandy-rtgwg-segment-routing-uloop-03, Apr. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-bashandy-rtgwg-segment-routing-uloop-03
  • [114] S. Hegde, P. Sarkar, “Micro-loop avoidance using SPRING,” Internet Engineering Task Force, Internet-Draft draft-hegde-rtgwg-microloop-avoidance-using-spring-03.txt, Jul. 2017, work in Progress. [Online]. Available: https://www.ietf.org/archive/id/draft-hegde-rtgwg-microloop-avoidance-using-spring-03.txt
  • [115] R. Geib (ed.) et al., “A Scalable and Topology-Aware MPLS Dataplane Monitoring System,” Internet Engineering Task Force, Internet-Draft draft-ietf-spring-oam-usecase-10, Dec. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-ietf-spring-oam-usecase/10
  • [116] N. Kumar (ed.), C. Pignataro (ed.) et al., “Label Switched Path (LSP) Ping/Traceroute for Segment Routing (SR) IGP-Prefix and IGP-Adjacency Segment Identifiers (SIDs) with MPLS Data Planes,” IETF RFC 8287, Dec. 2017. [Online]. Available: https://tools.ietf.org/html/rfc8287
  • [117] Z. Ali et al., “Operations, Administration, and Maintenance (OAM) in Segment Routing Networks with IPv6 Data plane (SRv6),” Internet Engineering Task Force, Internet-Draft draft-ali-spring-srv6-oam-01, Jul. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-ali-spring-srv6-oam-01
  • [118] Z. Ali et al., “Traffic Accounting in Segment Routing Networks,” Internet Engineering Task Force, Internet-Draft draft-ali-spring-sr-traffic-accounting-02, Jun. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-ali-spring-sr-traffic-accounting-02
  • [119] Z. Ali et al., “Bidirectional Forwarding Detection (BFD) for Segment Routing Policies for Traffic Engineering,” Internet Engineering Task Force, Internet-Draft draft-ali-spring-bfd-sr-policy-00, Mar. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-ali-spring-bfd-sr-policy-00
  • [120] D. Frost, S. Bryant, “Packet Loss and Delay Measurement for MPLS Networks,” IETF RFC 6374, Sep. 2011. [Online]. Available: https://tools.ietf.org/html/rfc6374
  • [121] S. Bryant et al., “UDP Return Path for Packet Loss and Delay Measurement for MPLS Networks,” IETF RFC 7876, Jul. 2016. [Online]. Available: https://tools.ietf.org/html/rfc7876
  • [122] R. Gandhi (ed.) et al., “Performance Measurement in Segment Routing Networks with MPLS Data Plane,” Internet Engineering Task Force, Internet-Draft draft-gandhi-spring-sr-mpls-pm-01, Jun. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/draft-gandhi-spring-sr-mpls-pm-01
  • [123] A. Bashandy et al., “Segment Routing interworking with LDP,” Internet Engineering Task Force, Internet-Draft draft-ietf-spring-segment-routing-ldp-interop-13, Jun. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-spring-segment-routing-ldp-interop-13
  • [124] C. Filsfils et al., “Segment Routing interoperability with LDP,” Internet Engineering Task Force, Internet-Draft draft-filsfils-spring-segment-routing-ldp-interop-03, Mar. 2015, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-filsfils-spring-segment-routing-ldp-interop-03
  • [125] P. Sarkar et al., “Anycast Segments in MPLS based Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-ietf-spring-mpls-anycast-segments-02, Jan. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-spring-mpls-anycast-segments-02
  • [126] P. Sarkar et al., “Anycast Segments in MPLS based Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-psarkar-spring-mpls-anycast-segments-02, Apr. 2016, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-psarkar-spring-mpls-anycast-segments-02
  • [127] C. Filsfils et al., “Segment Routing Recursive Information,” Internet Engineering Task Force, Internet-Draft draft-filsfils-spring-sr-recursing-info-05, Jun. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-filsfils-spring-sr-recursing-info-05
  • [128] H. Sitaraman et al., “Recommendations for RSVP-TE and Segment Routing LSP co-existence,” Internet Engineering Task Force, Internet-Draft draft-ietf-teas-sr-rsvp-coexistence-rec-04, May 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-teas-sr-rsvp-coexistence-rec-04
  • [129] H. Sitaraman et al., “Recommendations for RSVP-TE and Segment Routing LSP co-existence,” Internet Engineering Task Force, Internet-Draft draft-sitaraman-sr-rsvp-coexistence-rec-02, Feb. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-sitaraman-sr-rsvp-coexistence-rec-02
  • [130] X. Xu et al., “SR-MPLS over IP,” Internet Engineering Task Force, Internet-Draft draft-xu-mpls-sr-over-ip-01, Jun. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-xu-mpls-sr-over-ip-01
  • [131] S. Bryant et al., “MPLS Segment Routing in IP Networks,” Internet Engineering Task Force, Internet-Draft draft-bryant-mpls-unified-ip-sr-03, Oct. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-bryant-mpls-unified-ip-sr-03
  • [132] X. Xu et al., “Unified Source Routing Instructions using MPLS Label Stack,” Internet Engineering Task Force, Internet-Draft draft-xu-mpls-unified-source-routing-instruction-04, Sep. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-xu-mpls-unified-source-routing-instruction-04
  • [133] D. Voyer et al., “Insertion of IPv6 Segment Routing Headers in a Controlled Domain,” Internet Engineering Task Force, Internet-Draft draft-voyer-6man-extension-header-insertion-04, Jun. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-voyer-6man-extension-header-insertion-04
  • [134] K. Raza et al., “YANG Data Model for SRv6 Base and Static,” Internet Engineering Task Force, Internet-Draft draft-raza-spring-srv6-yang-01, Mar. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-raza-spring-srv6-yang-01
  • [135] S. Previdi et al., “Segment Routing Prefix SID extensions for BGP,” Internet Engineering Task Force, Internet-Draft draft-ietf-idr-bgp-prefix-sid-27, Jun. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-prefix-sid-27
  • [136] K. Patel et al., “Segment Routing Prefix SID extensions for BGP,” Internet Engineering Task Force, Internet-Draft draft-keyupate-idr-bgp-prefix-sid-05, Jul. 2015, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-keyupate-idr-bgp-prefix-sid-05
  • [137] S. Previdi et al., “Advertising Segment Routing Policies in BGP,” Internet Engineering Task Force, Internet-Draft draft-ietf-idr-segment-routing-te-policy-04, Jul. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-idr-segment-routing-te-policy-04
  • [138] S. Previdi et al., “Advertising Segment Routing Policies in BGP,” Internet Engineering Task Force, Internet-Draft draft-previdi-idr-segment-routing-te-policy-07, Jun. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-previdi-idr-segment-routing-te-policy-07
  • [139] A. Sreekantiah et al., “Segment Routing Traffic Engineering Policy using BGP,” Internet Engineering Task Force, Internet-Draft draft-sreekantiah-idr-segment-routing-te-00, Oct. 2015, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-sreekantiah-idr-segment-routing-te-00
  • [140] G. Dawra et al., “BGP Control Plane Extensions for Segment Routing based Service Chaining,” Internet Engineering Task Force, Internet-Draft draft-dawra-idr-bgp-sr-service-chaining-02, Jan. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-dawra-idr-bgp-sr-service-chaining-02
  • [141] G. Dawra et al., “BGP Control Plane for Segment Routing based Service Chaining,” Internet Engineering Task Force, Internet-Draft draft-dawra-spring-bgp-sr-service-chaining-00, Oct. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-dawra-spring-bgp-sr-service-chaining-00
  • [142] G. Dawra et al., “BGP Signaling of IPv6-Segment-Routing-based VPN Networks,” Internet Engineering Task Force, Internet-Draft draft-dawra-idr-srv6-vpn-04, Jun. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-dawra-idr-srv6-vpn-04
  • [143] G. Dawra et al., “BGP Signaling of IPv6-Segment-Routing-based VPN Networks,” Internet Engineering Task Force, Internet-Draft draft-dawra-bgp-srv6-vpn-00, Mar. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-dawra-bgp-srv6-vpn-00
  • [144] S. Previdi, K. Talaulikar, C. Filsfils, H. Gredler, and M. Chen, “BGP Link-State extensions for Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-ietf-idr-bgp-ls-segment-routing-ext-08, May 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-ls-segment-routing-ext-08
  • [145] S. Previdi et al., “BGP Link-State extensions for Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-gredler-idr-bgp-ls-segment-routing-ext-04, Oct. 2016, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-gredler-idr-bgp-ls-segment-routing-ext-04
  • [146] S. Previdi et al., “Distribution of Traffic Engineering (TE) Policies and State using BGP-LS,” Internet Engineering Task Force, Internet-Draft draft-ietf-idr-te-lsp-distribution-09, Jun. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-idr-te-lsp-distribution-09
  • [147] S. Previdi et al., “BGP-LS extensions for Segment Routing BGP Egress Peer Engineering,” Internet Engineering Task Force, Internet-Draft draft-ietf-idr-bgpls-segment-routing-epe-15, Mar. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgpls-segment-routing-epe-15
  • [148] S. Previdi et al., “Segment Routing Egress Peer Engineering BGP-LS Extensions,” Internet Engineering Task Force, Internet-Draft draft-previdi-idr-bgpls-segment-routing-epe-03, Apr. 2015, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-previdi-idr-bgpls-segment-routing-epe-03
  • [149] J. Tantsura et al., “Signaling Maximum SID Depth using Border Gateway Protocol Link-State,” Internet Engineering Task Force, Internet-Draft draft-ietf-idr-bgp-ls-segment-routing-msd-01, Oct. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-idr-bgp-ls-segment-routing-msd-01
  • [150] J. Tantsura et al., “Signaling Maximum SID Depth using Border Gateway Protocol Link-State,” Internet Engineering Task Force, Internet-Draft draft-tantsura-idr-bgp-ls-segment-routing-msd-05, Jun. 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-tantsura-idr-bgp-ls-segment-routing-msd-05
  • [151] L. Ginsberg et al., “BGP-LS Advertisement of IGP Traffic Engineering Performance Metric Extensions,” Internet Engineering Task Force, Internet-Draft draft-ietf-idr-te-pm-bgp-10, Mar. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-idr-te-pm-bgp-10
  • [152] G. Dawra et al., “BGP Link State extensions for IPv6 Segment Routing(SRv6),” Internet Engineering Task Force, Internet-Draft draft-dawra-idr-bgpls-srv6-ext-03, Mar. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-dawra-idr-bgpls-srv6-ext-03
  • [153] K. Talaulikar et al., “BGP Link-State Extensions for BGP-only Fabric,” Internet Engineering Task Force, Internet-Draft draft-ketant-idr-bgp-ls-bgp-only-fabric-00, Mar. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ketant-idr-bgp-ls-bgp-only-fabric-00
  • [154] S. Previdi et al., “IS-IS Extensions for Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-previdi-isis-segment-routing-extensions-05, Feb. 2014, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-previdi-isis-segment-routing-extensions-05
  • [155] S. Previdi et al., “Segment Routing IPv6 Prefix-SID,” Internet Engineering Task Force, Internet-Draft draft-previdi-isis-ipv6-prefix-sid-03, Nov. 2016, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-previdi-isis-ipv6-prefix-sid-03
  • [156] P. Psenak et al., “ISIS Segment Routing Flexible Algorithm,” Internet Engineering Task Force, Internet-Draft draft-hegdeppsenak-isis-sr-flex-algo-02, Feb. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-hegdeppsenak-isis-sr-flex-algo-02
  • [157] P. Psenak et al., “IGP Flexible Algorithm,” Internet Engineering Task Force, Internet-Draft draft-ietf-lsr-flex-algo-00, May 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-lsr-flex-algo-00
  • [158] J. Tantsura et al., “Signaling MSD (Maximum SID Depth) using IS-IS,” Internet Engineering Task Force, Internet-Draft draft-ietf-isis-segment-routing-msd-12, May 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-isis-segment-routing-msd-12
  • [159] J. Tantsura et al., “Signaling MSD (Maximum SID Depth) using IS-IS,” Internet Engineering Task Force, Internet-Draft draft-tantsura-isis-segment-routing-msd-02, Sep. 2016, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-tantsura-isis-segment-routing-msd-02
  • [160] L. Ginsberg et al., “Advertising L2 Bundle Member Link Attributes in IS-IS,” Internet Engineering Task Force, Internet-Draft draft-ietf-isis-l2bundles-07, May 2017, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-isis-l2bundles-07
  • [161] L. Ginsberg et al., “Advertising L2 Bundle Member Link Attributes in IS-IS,” Internet Engineering Task Force, Internet-Draft draft-ginsberg-isis-l2bundles-02, Feb. 2016, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ginsberg-isis-l2bundles-02
  • [162] S. Previdi et al., “IS-IS Traffic Engineering (TE) Metric Extensions,” IETF RFC 7810, May 2016. [Online]. Available: https://rfc-editor.org/rfc/rfc7810.txt
  • [163] P. Psenak et al., “OSPF Extensions for Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-psenak-ospf-segment-routing-extensions-05, Jun. 2014, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-psenak-ospf-segment-routing-extensions-05
  • [164] P. Psenak et al., “OSPFv3 Extensions for Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-psenak-ospf-segment-routing-ospfv3-extension-02, Jul. 2014, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-psenak-ospf-segment-routing-ospfv3-extension-02
  • [165] J. Tantsura et al., “Signaling MSD (Maximum SID Depth) using OSPF,” Internet Engineering Task Force, Internet-Draft draft-ietf-ospf-segment-routing-msd-14, May 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-ospf-segment-routing-msd-14
  • [166] J. Tantsura et al., “Signaling MSD (Maximum SID Depth) using OSPF,” Internet Engineering Task Force, Internet-Draft draft-tantsura-ospf-segment-routing-msd-01, Sep. 2016, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-tantsura-ospf-segment-routing-msd-01
  • [167] P. Psenak et al., “OSPF Segment Routing Flexible Algorithm,” Internet Engineering Task Force, Internet-Draft draft-ppsenak-ospf-sr-flex-algo-00, Feb. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ppsenak-ospf-sr-flex-algo-00
  • [168] S. Giacalone et al., “OSPF Traffic Engineering (TE) Metric Extensions,” IETF RFC 7471, Mar. 2015. [Online]. Available: https://rfc-editor.org/rfc/rfc7471.txt
  • [169] S. Sivabalan et al., “PCEP Extensions for Segment Routing,” Internet Engineering Task Force, Internet-Draft draft-sivabalan-pce-segment-routing-03, Jul. 2014, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-sivabalan-pce-segment-routing-03
  • [170] S. Sivabalan et al., “Carrying Binding Label/Segment-ID in PCE-based Networks.” Internet Engineering Task Force, Internet-Draft draft-sivabalan-pce-binding-label-sid-04, Mar. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-sivabalan-pce-binding-label-sid-04
  • [171] A. Natal et al., “LISP Control Plane for SRv6 Endpoint Mobility,” Internet Engineering Task Force, Internet-Draft draft-rodrigueznatal-lisp-srv6-00, Jul. 2018, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-rodrigueznatal-lisp-srv6-00
  • [172] “Linux Foundation Wiki - iproute2.” [Online]. Available: https://wiki.linuxfoundation.org/networking/iproute2
  • [173] “SRv6 - Linux kernel implementation - basic configuration.” [Online]. Available: https://segment-routing.org/index.php/Implementation/Configuratio
  • [174] D. Lebrun, “Reaping the benefits of IPv6 Segment Routing,” 2017. [Online]. Available: https://inl.info.ucl.ac.be/system/files/phdthesis-lebrun.pdf
  • [175] “SRv6 - Linux kernel implementation - advanced configuration.” [Online]. Available: https://segment-routing.org/index.php/Implementation/AdvancedConf
  • [176] “Linux Kernel Newbies - Linux 4.16.” [Online]. Available: https://kernelnewbies.org/Linux_4.16
  • [177] “Linux netfilter hacking.” [Online]. Available: https://www.netfilter.org/documentation/HOWTO/netfilter-hacking-HOWTO-3.html
  • [178] A. Abdelsalam et al., “SERA: SEgment Routing Aware Firewall for Service Function Chaining scenarios,” in IFIP Networking 2018.   IEEE, May 2018.
  • [179] “Arch Wiki - iptables.” [Online]. Available: https://wiki.archlinux.org/index.php/iptables
  • [180] “Linux Kernel Newbies - Linux 4.18.” [Online]. Available: https://kernelnewbies.org/Linux_4.18
  • [181] “A thorough introduction to eBPF.” [Online]. Available: https://lwn.net/Articles/740157/
  • [182] Programming network actions with BPF. [Online]. Available: https://segment-routing.org/index.php/Implementation/BPF
  • [183] SRv6-BPF. [Online]. Available: https://github.com/Zashas/Thesis-SRv6-BPF
  • [184] SRv6-BFD. [Online]. Available: https://github.com/Zashas/SRv6-BFD
  • [185] “What is VPP ?” https://wiki.fd.io/view/VPP.
  • [186] SRv6-net-prog. [Online]. Available: https://netgroup.github.io/SRv6-net-prog/
  • [187] “FRRouting,” available online at https://frrouting.org.
  • [188] Quagga project. [Online]. Available: https://www.quagga.net
  • [189] OSPFv2 Segment Routing. [Online]. Available: https://github.com/FRRouting/frr/blob/master/doc/developer/ospf-sr.rst
  • [190] P4 Language Consortium. [Online]. Available: https://p4.org
  • [191] P4 Runtime. [Online]. Available: https://p4.org/p4-runtime/
  • [192] Stratum Project. [Online]. Available: https://stratumproject.org
  • [193] OpenCORD Project. [Online]. Available: https://opencord.org
  • [194] SEBA Project. [Online]. Available: https://guide.opencord.org/profiles/seba/
  • [195] Trellis Tutorial. [Online]. Available: http://bit.ly/trellis-p4-slides
  • [196] S. Salsano et al., “OSHI-Open Source Hybrid IP/SDN networking (and its emulation on Mininet and on distributed SDN testbeds),” in Software Defined Networks (EWSDN), 2014 Third European Workshop on.   IEEE, 2014, pp. 13–18.
  • [197] S. Salsano et al., “Hybrid IP/SDN networking: open implementation and experiment management tools,” IEEE Transactions on Network and Service Management, vol. 13, no. 1, pp. 138–153, 2016.
  • [198] Open vSwitch. Available online at http://openvswitch.org.
  • [199] S. Salsano et al., “Mantoo-a set of management tools for controlling SDN experiments,” in Software Defined Networks (EWSDN), 2015 Fourth European Workshop on.   IEEE, 2015, pp. 123–124.
  • [200] PMSR Tutorial. [Online]. Available: https://github.com/netgroup/SDN-TE-SR-tools
  • [201] OSHI homepage. [Online]. Available: http://netgroup.uniroma2.it/OSHI
  • [202] B. Pfaff et al., “The Open vSwitch Database Management Protocol,” RFC 7047, Dec. 2013. [Online]. Available: https://rfc-editor.org/rfc/rfc7047.txt
  • [203] SRN. [Online]. Available: https://github.com/segment-routing/srn
  • [204] SRN Overview. [Online]. Available: https://segment-routing.org/index.php/SRN/SRN
  • [205] Using SRN. [Online]. Available: https://segment-routing.org/index.php/SRN/UsingSRN
  • [206] SRv6-SDN. [Online]. Available: https://netgroup.github.io/srv6-sdn/
  • [207] ROSE. [Online]. Available: https://netgroup.github.io/rose/
  • [208] SRv6Pipes. [Online]. Available: https://github.com/segment-routing/SRv6Pipes
  • [209] SRv6Pipes Walkthrough. [Online]. Available: https://segment-routing.org/index.php/SRv6Pipes/SRv6Pipes
  • [210] SRNK. [Online]. Available: https://netgroup.github.io/srnk/
  • [211] RobustlyDisjointPathsCode. [Online]. Available: https://bitbucket.org/franaubry/robustlydisjointpathscode
  • [212] C. Filsfils et al., “SRv6 interoperability report,” IETF, Internet-Draft, September 2018. [Online]. Available: https://tools.ietf.org/html/draft-filsfils-spring-srv6-interop-01
  • [213] Cisco System. [Online]. Available: https://www.cisco.com
  • [214] Barefoot Networks. [Online]. Available: https://www.barefootnetworks.com
  • [215] BMv2 Switch. [Online]. Available: http://www.bmv2.org
  • [216] Cavium. [Online]. Available: https://www.cavium.com
  • [217] Wireshark. [Online]. Available: https://www.wireshark.org
  • [218] tcpdump. [Online]. Available: http://www.tcpdump.org
  • [219] iptables. [Online]. Available: https://www.netfilter.org
  • [220] nftables. [Online]. Available: https://wiki.nftables.org/wiki-nftables/index.php/Main_Page
  • [221] snort. [Online]. Available: https://www.snort.org
  • [222] SRv6 Interop Demo. [Online]. Available: https://blogs.cisco.com/sp/segment-routing-ipv6-interoperability-demo-is-already-there
  • [223] Cisco Blogs, “Cisco Supports SoftBank on First Segment Routing IPv6 Deployment in Prep for 5G,” https://newsroom.cisco.com/press-release-content?articleId=1969030.
  • [224] Cisco Blogs, “Iliad Launches 5G Ready IP Network Architecture with Segment Routing IPv6 in Italy,” https://newsroom.cisco.com/press-release-content?articleId=1978361.
  • [225] “OpenStack.” [Online]. Available: https://www.openstack.org
  • [226] “Kubernetes.” [Online]. Available: http://kubernetes.io
  • [227] “Segment Routing - News.” [Online]. Available: http://www.segment-routing.net/news