A Lower Bound on the stability region of Redundancy-d with FIFO service discipline

by   Gal Mendelson, et al.

Redundancy-d (R(d)) is a load balancing method used to route incoming jobs to K servers, each with its own queue. Every arriving job is replicated into 2<=d<=K tasks, which are then routed to d servers chosen uniformly at random. When the first task finishes service, the remaining d-1 tasks are cancelled and the job departs the system. Despite the fact that R(d) is known, under certain conditions, to substantially improve job completion times compared to not using redundancy at all, little is known on a more fundamental performance criterion: what is the set of arrival rates under which the R(d) queueing system with FIFO service discipline is stable? In this context, due to the complex dynamics of systems with redundancy and cancellations, existing results are scarce and are limited to very special cases with respect to the joint service time distribution of tasks. In this paper we provide a non-trivial, closed form lower bound on the stability region of R(d) for a general joint service time distribution of tasks with finite first and second moments. We consider a discrete time system with Bernoulli arrivals and assume that jobs are processed by their order of arrival. We use the workload processes and a quadratic Lyapunov function to characterize the set of arrival rates for which the system is stable. While simulation results indicate our bound is not tight, it provides an easy-to-check performance guarantee.



There are no comments yet.


page 1


Stability for Two-class Multiserver-job Systems

Multiserver-job systems, where jobs require concurrent service at many s...

Improving the performance of heterogeneous data centers through redundancy

We analyze the performance of redundancy in a multi-type job and multi-t...

Blind GB-PANDAS: A Blind Throughput-Optimal Load Balancing Algorithm for Affinity Scheduling

Dynamic affinity load balancing of multi-type tasks on multi-skilled ser...

Stability, memory, and messaging tradeoffs in heterogeneous service systems

We consider a heterogeneous distributed service system, consisting of n ...

A Survey of Stability Results for Redundancy Systems

Redundancy mechanisms consist in sending several copies of a same job to...

Interference Queueing Networks on Grids

Consider a countably infinite collection of coupled queues representing ...

Achievable Stability in Redundancy Systems

We consider a system with N parallel servers where incoming jobs are imm...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Redundancy and cancellation based routing has attracted much attention in the last decade [1, 2, 3, 4, 5, 6, 7]. The basic motivation behind using redundancy and cancellation is reducing the tail of the job completion time distribution. The idea is to replicate a job and send its copies, referred to as tasks, to different servers for processing. When the first task is finished being processed the job is deemed complete and leaves the network. The premise is that allowing copies of a job to traverse different paths in the network makes it highly improbable that all of the copies experience large queuing delay and/or processing time.

Implementing such redundancy and cancellations mechanisms incurs an overhead which can include software, hardware, control, memory and computational power. The performance-cost trade-off of these schemes may well be worthwhile since the potential benefits in terms of performance is known in some cases to be substantial [1, 2, 3, 4, 5].

In this paper we are concerned with a specific scheme called Redundancy- (R()), used to route incoming jobs to servers, working at rate , each with its own queue. Within each server, service is given by order of arrival (FIFO). Every arriving job is replicated into tasks, which are then routed to distinct servers chosen uniformly at random. When the first task finishes service, the remaining tasks are cancelled and the job departs the system.

Our main research question is concerned with a first order performance criterion of R(): what is the set of arrival rates under which the R() system is stable? We refer to this set as the stability region. Here and throughout we refer to a system as stable

if the underlying Markov chain describing the load (e.g. queue lengths, workloads) in the system is positive recurrent.

For policies with no redundancy such as random routing or ‘join the shortest queue’, it is well known that the queueing system is stable as long as the arrival rate satisfies . For R(), denoting by the service time requirements of the tasks belonging to a single job, such that , the stability region is known exactly only in two special cases:

are independent and exponentially distributed. Then, the stability region remains

, for all values of [6].
(ii) (full redundancy). Then the stability region of R() is [7].

The only other closed form result the author is aware of is a lower bound on the stability region of R() such that if then the system is stable [7]. This lower bound is tight for but is limited in general because it does not depend on . Finally, the authors of [8] implicitly characterize the exact stability region in the case where are identical (i.e. ) and exponentially distributed. The stability condition is given in terms of the mean number of jobs in service in an associated ‘saturated’ system. Using this result, our lower bound can be easily derived for this special case.

Characterizing the stability region of the R() queueing system for

is a challenging problem. The main difficulty lies in the fact that the amount of work that enters the system upon a job’s arrival depends not only on its primitive service time requirement but also on the state of the system. This is due to the possibility of several servers working in parallel on tasks belonging to the same job, not even necessarily beginning at the same time. This results in a Markov chain with state-dependent transition probabilities, a difficult process to analyze.

In this paper we consider a discrete time system with Bernoulli arrivals and assume that jobs are processed by their order of arrival (FIFO). Our main result is a lower bound on the stability region of R() for a general joint service time distribution of tasks with finite first and second moments. Specifically, we allow a general dependence structure of the service time distribution of tasks belonging to the same job.

Informally, given , and the distribution of , let


Then if the system is stable. The significance of this result is that it provides a first order performance guarantee in terms of the arrival rates the R() system is able to support.

In the derivation of we use the fact that given that the system is initially in a balanced state (e.g. empty), the largest workloads must always be equal [7]. For , the quantity equals the probability that exactly of the servers with the largest workloads are chosen for a job’s tasks. The lower bound is derived by bounding the expected amount of incoming work for different values of .

Our bound coincides with the exact stability region for the two extreme cases of (no redundancy) and (full redundancy). Its relation to the only other known bound of depends on , and the distribution of . While no one bound implies the other in the general case, the dependence of on makes it valuable.

For example, after setting and using the fact that , we obtain

If we choose and then and we obtain

However, if we choose and , then and we obtain

Thus, taking the maximum of and yields a new and improved lower bound for the stability region of R() with FIFO service discipline.

Apart from the closed form lower bound, we make the following additional contributions. The first is a rigorous derivation of a workload model of the R() system. The second is a method of using the standard Lyapunov based technique for proving stability in a setting where the stability region is not a-priori known. The third is identifying a tighter lower bound on the stability region than , which we denote by , given as a solution to a certain minimization problem. In fact we prove that if then the system is stable, and then prove . Forth and final, we provide simulation results evaluating and comparing it to .

The rest of the paper is organized as follows. In Section II we rigorously derive a workload model for the system. In Section III we present our main results and their proofs. Section IV is devoted to simultion results.

We use the following notation. For write

. For two random vectors

and write for equality in distribution. Write for .

Ii Redundancy routing model

Consider a time slotted system with a single dispatcher and homogeneous servers. Each server has an infinite size buffer in which a queue can form and the servers do not idle when there is work in the buffer. Each server completes a single unit of service when it has work to do and service is given by order of arrival, i.e. FIFO. We assume a server can work on a job that has just arrived.

Arrival At each time slot , a job arrives to the dispatcher with probability

, according to the value of a Bernoulli random variable (RV)

, such that .

Routing When a job arrives, the dispatcher immediately sends replicas of the job, where , to distinct servers. We refer to these replicas as tasks. Denote by the set of all -sized subsets of . For each , denote by a set-valued RV taking values in with equal probability. If a job arrives at time slot then determines which servers will receive its tasks. We assume that are independent and identically distributed (i.i.d) across time slots. When the first of the job’s tasks finishes service, the remaining tasks are immediately cancelled and this marks the job’s departure time from the system. For completeness, in the case where tasks are completed at the exact same time, we refer to the task in the smallest indexed server as completed and to the rest as cancelled.

Service time distribution. If a job arrives at time slot , the service duration requirements for its tasks are determined by the random vector whose members take values in . The quantity represents the service time requirement of the task that is to be sent to server , provided it is a member of . Denote .

The homogeneity of the servers is captured by the assumption that the distribution of

is symmetric with respect to its members, such that the joint distribution of any subset of

coincides with any other subset of the same size. Formally,

Assumption II.1 (Homogeneity).

For every , and such that and respectively, we assume that


For example, direct consequences of this assumption are and . We also assume and .

We further assume that are i.i.d. across time slots. As a consequence, the service time requirements of jobs are i.i.d. However, we make no additional assumption on the distribution of for each . Therefore the service requirements of tasks belonging to the same job may be dependent. Finally, the arrival and service processes are assumed to be independent.

Time scaling. To avoid the possibility of being inside the stability region, we scale time appropriately. This translates to an assumption on as follows. The average amount of work that enters the system upon a job’s arrival is bounded from below by the average of the minimum of the service requirements of its tasks, i.e. . Note that it does not matter which members we take due to Assumption 1. The servers clear at most units of work, so we require


such that the largest possible for which the system is stable is strictly smaller than 1.

Workload. Denote by the workload in buffer at time slot , after a possible arrival and service. This is defined as the amount of time it will take (in time slots) for the existing tasks in the buffer (including the one in service if there is any) to leave the system. Note that tasks can leave the system either due to service completion or due to cancellation. Also, due to the FIFO service discipline, the workloads depend only on present tasks in the buffers and not on future arrivals. This is not true for other service disciplines such as last-in-first-out or processor-sharing.

Denote and . We refer to as the workload process. We assume that the system starts empty, i.e.


Markov chain formulation. We turn to analyze the dynamics of the workload process. To this end, denote


and let denote the indicator RV which equals if server is a member of the chosen servers at time slot . Then

Proposition II.2.

For each we have


The first part of (6) is a standard balance equation. The workload at time equals the workload at time plus arrival minus service, and is kept non-negative. The second part of (6) is less trivial and captures the complexity of the R() model with FIFO service discipline.

The quantity represents the total amount of work server receives at time slot provided a job arrives. If then and we have . Otherwise, the amount of work that server receives depends on the workload in the other members of , as well as the service time requirements of the arriving job’s tasks.

Considering the FIFO service discipline within each server and the definition of the workloads, the task that reaches a server first out of the tasks is the one that is sent to the server with the least workload. Considering the cancellation mechanism, the task that finishes processing first out of the tasks is the one that is sent to server for which is minimal. Denote this server by , where in the case of several minimizers, the smallest index is returned.

If , or, written differently using (4), , the task that arrives to server will be cancelled before being processed and . If , then by the definition of the task in server will be cancelled at the exact same time server completes its task. Thus, the workload at server will be truncated and will equal and the amount of work server receives equals

Note that may be negative. Overall, we obtain


To connect (8) with (6), we argue that


Indeed, if , the right hand side of (9) is zero, and since , the left hand side of (9) equals zero as well. If , by the definition of , for , we have

which completes the proof. ∎

Define the state space of as all members of that can be reached from an empty state. Equations (3)-(6) uniquely define the process as a Markov chain on . Since , all states in communicate and the empty state has a self transition. Therefore, is irreducible and a-periodic.

Remark II.3.

Since we have assumed the system starts empty (3), by Property 1 in [7], the largest workloads are always equal. Thus the largest components of any member of must be equal.

Remark II.4.

The R() policy routes tasks to servers chosen uniformly at random. It does not use workload or service time information, which we only use for modelling and stability analysis.

Iii Main results

We first state our results, then discuss them in detail. The proofs immediately follow.

Iii-a Statements

Our first result identifies a non-trivial lower bound on the stability region, given as the solution of a certain minimization problem. This lower bound, which we denote by , satisfies that if then is positive recurrent. Our second result, which is the main result of this paper, is a closed form formula for a lower bound on the stability region, satisfying . To this end, define the space of ordered states in by


and define the space of vectors capturing the differences between the coordinates of members of by


Note that . For , define


For ease of notation, denote



Proposition III.1.

If then is positive recurrent.

We now state the main result of this paper. To this end, for , define


A simple calculation yields



Theorem III.2.

Let and defined as in (15) and (18) respectively. Then

Specifically, if then is positive recurrent.

Iii-B Intuition and discussion

Before proving Proposition III.1 and Theorem III.2, we discuss (15) and (18) in detail.

Intuition for . The basic idea is as follows. By (6), the average amount of incoming work at time slot is given by


This quantity depends on the state at the end of time slot only through , i.e. the difference between the workloads. We require that for all relevant values of the right hand side of (21) is less than or equal to and find the largest for which this still holds. This is a notion of sub-criticality. The challenge is to prove that this is sufficient for stability for the R() system.

The infimum in (15) is taken over and not for two reasons. First, by the symmetry of the servers, it is sufficient to consider only ordered states in of the form , i.e. , whenever . Second, all states reached by must have that the largest workloads are equal, namely for .

Intuition for . Suppose a job arrives and corresponding tasks are sent to distinct servers. Further suppose that exactly of the servers with the largest workloads are chosen, where . For simplicity, assume that the chosen servers are , such that and that the servers have the largest workload in the system, implying . Then, server receives at most units of work, server 2 at most and up to server receiving at most . The last servers receive at most . Taking expectation and summing over possible values of gives an upper bound on the expected amount of work that enters the system upon a job’s arrival, which yields the closed form expression of in (18).

Why we included in this paper. One can prove that is a lower bound on the stability region without resorting to at all. This requires a minor modification of the proof of Proposition III.1 and some elements from the proof of Theorem III.2.

However, we feel that is interesting in its own right. Indeed, given , and the distribution of , if one can solve for numerically then one may obtain a tighter bound than . But, this is not trivial. First, the set is infinite. Second, different distributions of may change the set of possible states the system can reach resulting in different sets which, in turn, may be difficult to characterize. Third, for each member of , one must explicitly calculate the expected value of the minimum of several functions of . This may be computationally expensive for certain distributions. We leave this as an open problem.

The special case of (no replication). In this case and every incoming job is randomly routed to one of the servers with equal probability. Together with the fact that by definition , Equation (21) reduces to

and the solution to (15) is given by as expected. As for , substituting in (18) and using the convention that yields .

The special case of (full replication). In this case the workloads of all of the servers are equal and all servers are chosen for each job. Thus, , and by (18), as expected.

Example calculation. Consider the case where , and the joint service time distribution satisfies that are i.i.d and

where and . By (17) we have and a straightforward calculation yields

Thus, by (18) we have

We use the following two results in the proofs that follow.

Proposition III.3 (Balance).

In a R() system, with initial condition, the largest workloads are equal at all times.


The proof is given in Property 1 in [7]. ∎

Proposition III.4 (Average workload).

Consider a R() system with servers. Let such that . Let . Denote





Relation (21) is an immediate consequence of (6) and (14). We now prove that for any we have and the relation (22) follows. This simply means that the average incoming workload is monotonic non-increasing when considering servers ordered by their workload.

Consider the four possibilities describing whether or not and are members of . Under the event that and are not in , both receive zero work. If both are in , then cannot receive more work than due the the minimum taken in (6). Finally, for any event under which is chosen and is not, there is an event with equal probability where is not chosen and is, and the rest servers stay the same, and vice versa. Again, by (6), the amount of average work that enters server under the first event is no less than the amount of average work that enters server under the second event. This concludes the proof. ∎

Iii-C Proofs of main results

Proof of Proposition III.1. Since is irreducible and aperiodic, by Theorem 3.3.7 of [9], it suffices to prove that if , then a Lyapunov drift condition holds. Namely, that there exist a function , a finite set and constants , such that


where denotes the drift at time slot , namely


While we assumed in (3) that the system starts empty, with a slight abuse of notation and for simplicity, in what follows we suppress the dependence on by writing and instead of and , respectively. We choose the quadratic function


Since the members of have finite first and second moments, it is trivial that the left hand side of (23) is bounded from above by some positive constant , uniformly over all , for any finite set . We omit the details.

Next, we choose the finite set to be of the form


where is a large enough constant whose value is determined later in the proof.

Consider a state . By the homogeneity of the servers, the symmetric distribution of and the uniformly at random routing choice, we can, without loss of generality, consider such that

By the definition of in (26), we have


and by Proposition III.3,

By (6), using the notation in (14) and denoting , we have

and therefore


Using (21), (24), (25) and (28) we obtain


where the constant satisfies


The existence of is due to the finite first and second moments of the members of .



The argument proceeds by analyzing . First, by (5) and (22),

Second, since , there exists such that . Thus

where in the last inequality we have used the definition of in (15). Denote by the lowest index in such that , namely






Next, we argue by induction that


with the convention that . Inequality (37) holds trivially for . Suppose it holds for . Then

where the first inequality is due to the induction hypothesis and the second is due to (35). Taking in (37) yields

and therefore


By the definition of in (34) and by (36), the coefficients , multiplying respectively, are strictly negative. Define


By (38) and (40) we obtain


where in the last inequality we have used (27). Combining (29), (33) and (41) we obtain

where we recall from (40) that .

Finally, we determine and in (23). Given the primitive arrival and service processes, the constants in (32) and in (40) are given. Fix some and choose (which defines ) to be large enough such that . This concludes the proof.

Proof of Theorem III.2. Recall that by (15)

where is given in (11). Denote by the th member of such that


With this notation at hand we can write


Proof that . Taking only the first term of the right hand side of (43) yields


By the definition of , and in (11), (13) and (42), respectively, we have that . Therefore