1. Introduction
Cellular channels are known to fluctuate rapidly over short periods of time (zaki2015adaptive). 3G and LTE network measurements (Nikravesh; Huang2013; Hu2015) demonstrated that variations in the channel cause significant performance differences across carriers, access technologies, geographic regions, and time. Rapid channel fluctuations cause lossbased congestion control (CC) algorithms to overreact and underperform (cardwell2016bbr), resulting in bufferbloat and high delays (gettys2011bufferbloat; Guo2016Understanding; Jiang2012). Several protocols such as Sprout (winstein2013stochastic), Copa (copa), Verus (zaki2015adaptive) and BBR (cardwell2016bbr) have demonstrated significant performance gains against traditional TCP variants over highly variable network channels. A common recurring theme across these protocols is to use delaybased signals to measure the network congestion state. While there is a broad array of research on the dynamics of lossbased CC protocols (padhye1998modeling; cardwell2000modeling; sun2019model), we still lack a principled framework for understanding the dynamics of delaybased protocols.
This paper proposes ModelDriven Interpretability (MDI) CC framework, aiming to enhance the ability to interpret delaybased CC protocols’ behavior. Given any protocol, the MDI framework uses empirical data on the protocol’s performance for training a stochastic twodimensional discretetime Markov model to represent the protocol’s behavior. In essence, using the empirical behavior of a protocol across diverse network conditions, MDI converts a protocol into a stochastic random walk in Markovian state space. Each state transition is determined by the delay variation feedback from the network. MDI aims to:

[noitemsep,topsep=0pt,parsep=0pt,partopsep=0pt]

Closely approximates the mean/variance of the throughput and delay distributions of the original protocol.

Track the original protocol’s temporal behavior, i.e., how to react to variations in network conditions.
We note that achieving these two properties for a broad array of protocols is a nontrivial task. In the MDI framework, the notion of protocol memory is implicitly captured in the definition of the state space (transition probabilities), and the stochastic random walk using delay feedback. While the state space represents a significant approximation to the original protocol, we show that MDI successfully approximates the protocol behavior in practice.
To evaluate MDI, we developed MDI versions of two different protocols: Verus (zaki2015adaptive) and Copa (copa). Using realworld cellular traces in 3G and 4G networks and across synthetic highly variable network conditions, we show that the MDI version of a protocol closely approximates the throughput and delay distributions of the original protocol and temporally tracks the protocols’ behavior. We demonstrate two specific benefits of MDI in this paper:
Visualizing Protocols: A protocol statespace representation enables visual understanding of its behavior, including measuring how state transitions vary across: (i) protocols under the same network condition; (ii) network conditions under the same protocol.
Reasoning about Convergence: By representing a protocol in a Markovian state space, one can derive the mixing time and the corresponding stationary distribution of the MDI version of a protocol that we show empirically to mirror the protocol’s measured statistical properties closely.
As presented in this paper, the MDI framework is a smaller part of a much larger puzzle of understanding the properties of delaybased control protocols. This paper has primarily shown the feasibility of the MDI framework in modeling two such protocols using a Markov Model representation. One longterm motivation to use a Markovian framework is to leverage the vast body of statistics literature on Markov models and random walks to understand the stability, dynamics, and adaptivity of delaybased protocols. While we have shown initial empirical evidence for analyzing convergence properties of protocols and visualizing protocols using MDI, a detailed statistical analysis of protocols is necessary for future work. It is beyond the scope of this paper.
2. MDI Design
The main idea of MDI is to build a model that reflects the statistical properties, providing a more intuitive and predictable understanding of the protocol behavior. At an abstract level, MDI assumes that CC protocols can be modeled by the relationship between the current and the next state, where each state is a tuple of the relative change in the network delay and the sending window size.
2.1. Modeling Delay based Control
Consider a protocol
that uses delayvariations as a congestion signal. One can imagine such a protocol maintains a recent history of delay observations, which can be used to estimates the next sending window or rate. Let us consider an epoch as the unit of time for making a decision, which can be a variable or a fixed period depending on the protocol.
The challenge in a Markov model representation of a protocol is determining the appropriate state space and mapping the protocol actions to transitions within the states. The most straightforward approach is to map the absolute values directly by describing a state as (,) where and are the experienced delay and sending window in an epoch , respectively. We use and (without the epoch subscript ) to abstractly represent the observed delay and window parameters for brevity. While a twovariable state space using is simple, it may not be rich/generic since it may not be sufficient to capture the variations in these parameters. Suppose one were to represent the state space using a history of delay and window measurements. In that case, the state space representation could be much more vibrant but correspondingly much harder to learn accurately. In fact, for each additional dimension in the state space, we need an order of magnitude more training data to determine the state transitions. To capture the variations of the delay and window in the state space, we also consider (1) relative change in the delay across neighboring epochs (captured by ); (2) relative change in the window across adjacent epochs (obtained by ). These four parameters provide a richer representation of the state space. However, the training data required for the 4dimensional space is at least two orders of magnitude more than the space. To balance between state complexity and state richness, we chose to condense these four parameters into two composite parameters as and . By representing the delay and window in log space and quantizing the values (described in Section 2.2), we can better delineate variations in relative delay (or window) changes in comparison to variations in the actual delay (or window) values across different buckets in the state space. The quantization of these values also helps maintain a condensed twoparameter representation of the four parameters: window, delay, a relative change in window size, and the relative change in delay across epochs. We note that one can choose alternate statespace representations for the MDI framework; the key requirements are to balance the number of quantized states in the state space to capture protocol dynamics across different network conditions.
2.2. Discretetime Markov Model States
A discretetime Markov model of a protocol is represented in the form of a statetransition probability matrix. The matrix describes transition probabilities from one state to another obtained by training a protocol on a large set of network configurations. We call this the training phase of the Markov model. To capture the protocol behavior, the matrix should include as many states as the ones observed during the training. The state is defined as a tuple with value pairs of . Where and are calculated using the current epoch’s packet delays () and sendingwindow () and the previous epoch’s delay () and sendingwindow ():
(1) 
(2) 
Assume that a protocol adjusts the congestion window as a function of delay feedback. A user executing protocol has currently the following values: the current sending window , and the previous epoch delay feedback . To decide on the value of the next window , the user has to first identify the current delay . The protocol decides the next window based on the following factors: the prior window , and the delay variations. Only upon observing , would be aware of the true represented state in the model space of the protocol. Essentially, given an initial window and delay , the protocol has three variations that influence a transition from to : (i) the variation in the initial observation ; (ii) the variation in the decision making of ; (iii) the variation in the next delay observation . Note that, it is not necessary for two users running the same protocol and in the same state , to derive the same next window . This decision is influenced by two factors: (i) different windows/delays values could effectively arrive at the same model state ; (ii) different flows may observe variations in prior observations of delays and windows.
2.3. Deriving the MDI Transition Matrix
The key assumption that MDI makes is that the state transition from to can be captured by a guided Markov model with two basic properties: the delay feedback guides the direction of the window change (increase or decrease), and the delay variations of and have an inherent randomness that influence the protocol choice of the next window . The guided Markov assumption is clearly an approximation of the original protocol behavior. To derive the transition matrix, we use a protocol emulation strategy in a constrained network environment. Consider a network simulation environment where one can execute the protocol under various network conditions and background traffic. Our setup’s network environment is defined by a set of network traces that specify bandwidth, packet loss, and RTT variations. The protocol can be executed by simulating network flows executing the protocol in the presence of competing traffic. We perform a broad array of network simulations by varying the network traces and the background traffic emulating several realworld protocols, including . For each simulation, we measure the state transitions of across the model states. By observing all possible state transitions of , with ranging from to , and ranging from to
, a 2D Markov chain is created defining the following states: current state
and next state , where and are the current state indexes of and , respectively. Similarly, and represents the next state indexes. To reduce the state space of possible values for , we quantize these values into small buckets. MDI captures the state transitions in the form of a transition probability matrix written as:(3) 
Thus, defines a specific row in the transition matrix. Depending on next value, represented by , we obtain a subset of values from this specific row (i.e., the probability going to any of the possible in the state .
2.4. Model Training Methodology
This paper focuses on training two delaybased protocols: Verus and Copa. The training is performed over a large sample of cellular traces covering a wide range of diverse scenarios. We ran each protocol through a network emulator over a large set of traces randomly synthesized from the training traces. The protocol behavior is captured by logging the set of congestion windows and their experienced correlated delays in each run. Next, the logged window and delay values are quantized (Equation 1 and 2). The quantized values are used to obtain the transition probability matrix where each state is the quantized pair . The matrix is structured in quadrants, highlighted by yellow and green in Figure 1.
Each quadrant represents a particular current delay on the yaxis and a next delay on the xaxis, these values are quantized in the range to to keep the matrix from being prohibitively long. Each quadrant is further divided into smaller chunks representing the current values of on the yaxis and a next window variable on the xaxis, which are quantized in the range to . Figure 1 shows an empty sample matrix. The transition probability for each chunk is computed by counting the number of occurrences of going from one state to another as . We normalize each row within a quadrant so that all outgoing transition probabilities of any state would sum to 1.
2.5. MDI Implementation
We implemented a generic sender and receiver in C that takes a transition matrix as an input and uses the matrix to decide the next sending window. The sender uses UDP as the transport protocol. It includes calculating the packet delays based on the incoming ACKs and uses the delay to determine the sending window size after each epoch. Epoch time is when the algorithm updates the congestion window. Algorithm 1 outlines the MDI control loop. The model algorithm identifies the next sending window in every epoch, obtained from the transition matrix, where a row within a quadrant of the matrix represents all possible values for the future sending window. MDI first identifies the operating quadrant through the row and column index. The row index is taken from the previous delay , and the column index from the current delay (inferred from the incoming ACKs). Once the operating quadrant is identified, a particular row within the quadrant can be determined by the previous sending window . This row represents all possible sending windows decisions for the next epoch, each associated with a specific probability value. To decide the next sending window, MDI draws a random number (between 0 and 1) to determines the closest matching sending window. This process is a guided randomwalk within the state transition probability matrix. If the values are outside the matrix dimensions, the next sending window is determined by a multiplicative increase/decrease to the current window to force it back to the matrix bounds ( and ).
3. Evaluation
We evaluated two CC protocols as a proofofconcept of MDI: Verus, and Copa. These protocols are modeled through the training phase by generating the model transition matrix. The training is done using a set of 1000 different cellular traces (collected from realworld 3G/4G networks) that cover a wide range of network scenarios. To replay these traces, we used the MahiMahi (netravali2015mahimahi) linkshell network emulator. We used a different set of cellular traces for testing, taken from several previously published papers:

[noitemsep,topsep=0pt,parsep=0pt,partopsep=0pt]

4G Verizon: taken from (winstein2013stochastic) and represents a recorded channel over Verizon’s 4G network in the US.

Highway: taken from (zaki2015adaptive), it represents a channel over a 3G network in the UAE while driving on a highway.

Rapidly changing network: inspired by (dong2015pcc), this trace represents a network with a highly fluctuating channel, where the capacity varied randomly every 5 seconds.
We wanted to evaluate how well a model representation of an algorithm can track the throughput and delay of the native algorithms when run on the same network traces. This section shows the results for the MDI versions of Verus and Copa. For each protocol, we demonstrate the temporal variations of the original protocol against the MDI version of the protocol for a snippet of a second run in one of the three scenarios in Figure (a)a, (a)a, (a)a, (a)a, (a)a, and (a)a
. The results show that across both protocols (Verus and Copa), the MDI models (represented in red) are able to accurately track the throughput of the native protocol (represented in blue) temporally. In addition, the MDI models are able to temporally track the delay behavior of these protocols. To quantify that the MDI models statistically matches the characteristics of the original protocols, we computed the Probability Density Function (PDF) of the throughput and delay for both Verus and Copa respectively. Figure
(b)b, (b)b, (b)b, (b)b, (b)b, and (b)b shows the PDF comparisons. It can be seen that the MDI throughput distributions match the native ones perfectly.In summary, we observe that MDI have the ability to accurately track the throughput behavior of the two used protocols across highly variable network conditions, evident by the results of Figure (c)c, (c)c, (c)c, (c)c, (c)c, and (c)c. The figures show the overall summary comparing different values of the results population. Each of the MDI model and the native protocol is depicted by a circular shape representing the operational region of the protocol circumscribed by the 25% and 75% percentile of the obtained throughput and delay, where the crosses (x) indicate the median values. The lower and upper part of the shape represents 25% and 75% of the throughput, respectively (yaxis), whereas the left and right part of the shape represents the 25% and 75% of the delay, respectively (xaxis). Results show that MDI is capable of achieving quite similar statistical performance in terms of delay and throughput with a slight delay penalty not exceeding 5% (i.e., in the rapidly changing channel).
4. Related Work
CC for cellular networks: conventional lossbased TCP variants, in particular Cubic (ha2008cubic), performs poorly in cellular networks. This is due to the high sending rate that fills up the buffers causing a bufferbloat (gettys2011bufferbloat). Bufferbloat is detrimental to the performance of delaysensitive applications like video calling. This has led to newer delaybased CC protocols like Sprout (winstein2013stochastic), and Verus (zaki2015adaptive) that are specifically designed in the context of cellular channels. Sprout focuses on reducing selfinflicted queuing delays, and Verus creates a balance between the packet delays and the throughput. Recently, PCC Vivace (dong2018pcc), which followed PCC (dong2015pcc)
, has shown to react well to changing networks while alleviating the bufferbloat. PCC Vivace leverages ideas from online (convex) optimization in machine learning to do rate control. LEDBAT
(RFC6817) is another delaybased CC algorithm developed for BitTorrent and other bulktransfer applications that had limited adoption. BBR (cardwell2016bbr) was recently proposed by Google and has shown promising results over cellular networks. BBR uses the bottleneck link’s round trip propagation and bandwidth to find CC’s optimum operating point.Applying machine learning to CC: new CC protocols being proposed have complex control loops, which makes them harder to understand in the context of different network conditions. The recent development of CC protocols that employ machine learning (e.g., Remy (remy), Vivace (dong2018pcc) and Indigo (yan2018pantheon)) have only compounded this issue (e.g., some of Remy’s CC protocols employ rule tables with more than 100 rules). Weinstein et. al. (Remy) (remy), Sivaraman et. al. (sivaraman2014) and Pötsch (potsch2016future) have provided different methodologies to model nonlinear CC from a theoretical perspective.
Analyzing TCP behavior: TCP and its variants have been thoroughly studied using the modeling, and analytical techniques (olsen2003stochastic; samios2003modeling; wierman2003unified; padhye1998modeling; cardwell2000modeling). A recent work called ACT (sun2019model) uses the concept of a guided random walk in the state space of implementation variables to find regions where the algorithm should never go, thereby indicating the existence of a possible bug in the implementation. Others also follow this approach of an automated modelguided method as well (jero2018automated) to explore the variable space in the implementation of a CC algorithm. Our modeling approach also uses a random walk, but our state space is limited to a delay and window variable, and our goal is not to reach unreachable points but to guide the model to follow the native algorithm it is modeling.
5. Discussion: Why MDI?
5.1. Visualizing Protocols
The MDI transition matrix helps reason about the essence of the CC protocol behavior. These matrices represent the probability distributions across the transition space; it highlights which states the protocol mostly operates in. It also shows how the protocol is likely to behave under specific network changes, such as increased or decreased network delay. Verus and Copa’s transition matrices (Figure
(a)a and (b)b) clearly show that the Verus matrix is less dense than Copa’s, which means that Verus takes more decisive actions compared to Copa that tend to explore more. Each protocol shows a particular pattern reflecting the protocol’s behavior; we call this the protocol fingerprint. The sectors in the matrix represent different transitions for a specific change in packet delay. The relative delay and window ranges are determined from the training phase (the 1% and 99% of the observed increase/decrease population).The protocols’ fingerprints reveal different characteristics of the protocol and how it reacts to various network changes. For example, the Verus transition matrix generally shows two distinct recurring patterns in the sectors: one on the left side of the matrix and the other on the right side. We can see that the right side pattern mainly contains window decrease probabilities. This is consistent with Verus’s design, where if the observed delay increases, Verus lowers the sending rate by moving the operation point down the delay profile curve.
However, the left side pattern consists mainly of a diagonal line from the upper left corner down to the lower right corner. Additionally, the pattern also has an antidiagonal, which becomes more dominant, moving down the sectors (i.e., when the delay feedback increases). This gives another insight to Verus. If a decrease in the previous delay is observed, it tends to continue alongside the same previous decision, extending the last window to decrease or increase. However, suppose Verus finds a delaydecrease with a prior increase in the delay. There is a higher probability that it might increase the window in the next decision despite the window decrease in the previous epoch. This confirms Verus’s exploration behavior, where, in case of a delay reduction, it tends to increase the window to explore the channel variations immediately.
On the other hand, Copa’s transition matrix shows that the matrix’s right side sectors show almost the same pattern, with substantial probabilities in the upper left and lower right corners of the sectors and nearly no values in the top right or lower left edges. This means that regardless of the previous delay values or the severity of the observed delay values’ increase, Copa tends to repeat its last epoch decision. For example, if Copa reduces the window, it will continue doing so in the next epochs. This is unlike Verus, where it tends to minimize the window in case of an observed delay increase. Looking at Copa’s matrix’s left sectors, we see that it has a similar pattern to the right side sectors with additional values in the upper right corner. These values become less dominant when moving down from the top to the bottom sectors. The sector’s upper right corner represents increasing the window despite a reduction in the previous epoch. Like Verus, Copa tends to increase the window by observing a delay reduction, and the severity of exploring increases when the previously observed delays are decreasing.
5.2. Convergence
Using our Markov formulation, we can provide convergence guarantees as strong as the original protocols, using properties of convergence of Markov chains. Before presenting our results, we briefly review some necessary notations and definitions regarding Markov chains and convergence.
Markov chains and Mixing times: Every Markov chain can be represented as a transition matrix , where the entry represents the probability of transitioning to state from state . Suppose
is row vector that represents a probability distribution over the state space at a time
. Then at , the distribution over the state space is given by . If the initial distribution at is given by , then we have from above that . The limiting distribution is the limit of as . If a unique limiting distribution exists, then it equals the stationary distribution, which is the row vector , such that. It is computed as the left eigenvector of the transition matrix corresponding to the largest eigenvalue
(Norris1997). The mixing time of a Markov chain, , is the time to convergence from an initial distribution , i.e., when the probability distribution over the state space is sufficiently “close” to the stationary distribution that they are indistinguishable from one another. Any random walk process in a finite Markov space is associated with a finite mixing time (aldous1983random). To obtain a conservative estimate, we define mixing time as the maximum convergence time starting from all possible initial states.Observations: In our context, the state space comprises of the Cartesian product of 11 states in the delay space and 21 states in the window space , a total of 231 tuples. If the start state is , then the initial distribution is a onehot vector, with 1 at the location corresponding to state and 0 everywhere else. Then, at every iteration (equivalent to an RTT), we compute , and declare convergence at time when the maximum elementwise difference between and is less than a certain defined threshold (). We compute mixing times for three different thresholds: , and . The last is chosen as it approximately equals the machine epsilon for 32bit float. Table 1 shows the mixing times (in RTTs) obtained from the transition matrix for both protocols.
The heatmaps in Figure 9 show the theoretical stationary distribution computed using the Markov chain transition matrix trained over a training sample of 1000 traces, compared with the empirical distribution of states after convergence (i.e., the mixing time) of the original protocols and the model versions over a separate testing sample of 60 cellular traces. The heatmaps are displayed over the twodimensional state space. The fact that these distributions match very closely is a robust result that our Markov model versions of the protocols are very close approximations of the original protocols. Table 2
shows the closeness of the two distributions in terms of the KullbackLeibler Divergence
(Kullback1951) of the two distributions. The KL Divergence is a measure of how well one distribution approximates another. The closer the KL Divergence is to zero, the better the approximation. The table also additionally shows a simple maximum elementwise absolute difference between the two distributions. From the heatmap plots and these numbers, we observe that the model allows us to analyze the original protocols’ convergence properties, which are a challenging proposition for delaybased protocols due to complex nonlinear control loops.Protocol  

Verus  24  55  85 
Copa  8  24  41 
Testing protocol  

Copa  0.017  0.004 
Model Copa  0.147  0.01 
Verus  0.101  0.02 
Model Verus  0.773  0.054 
6. Conclusions
This paper describes the MDI framework that can approximate delaybased protocols’ behavior and potentially help visualize protocol behavior, understand convergence properties, and derive a modelbased protocol replacement. We hope that this Markov modeling approach provides a new lens for understanding delaybased congestion control algorithms’ behavior on highly variable networks. In future work, we hope to extend this framework to understand the behavior of a broader array of protocols, analyze fairness properties of MDI protocols and explore alternative statespace protocol representations within MDI.
Comments
There are no comments yet.