Recent advances in technologies of smart devices, sensors and embedded processors have enabled the deployment of a large number of computational and storage resources in a physical space. We collectively call such a set of resources/ services a distributed service network. In a distributed service network, for example, Internet of Things [atzori10], these resources are heavily accessed by various end users/applications that are also distributed across the physical space.
An important design problem in such service networks is the assignment of users/applications to appropriate servers/ resources. Often servers have limited computational capabilities and can only serve a handful number of users. Hence for such a system with very large number of users and servers, an appropriate server selection strategy typically involves minimizing the maximum number of users assigned to a server, also known as the maximum load. While the optimal server selection problem can be solved centrally, due to scalability concerns, it is often preferred to relegate the assignment task to individual users themselves. This interpretation leads to formulating a randomized load balancing problem for the distributed service network with the goal to make the overall user-to-server assignment as fair as possible. Many previous works [Adler1998, mitzenmacher96] have used randomization as an effective tool to develop simple and efficient load balancing algorithms in non-geographic settings. A randomized load balancing algorithm can be described as a classical balls and bins problem as follows.
In the classical balls-and-bins model of randomized load balancing, balls are placed sequentially into bins. Each ball samples bins uniformly at random and is allocated to the bin with the least load with ties broken arbitrarily. It is well known that when and this assignment policy results in a maximum load of
with high probability. However, if, then the maximum load is w.h.p. [Azar99]. Thus, there is an exponential improvement in performance from to . This policy with is widely known as Power of Two (POT) choices and the improvement in maximum load behavior is known as POT benefits [mitzenmacher96].
While classical balls and bins based randomized load balancing can directly be used for user-to-server assignment in a geographic setting, it is oblivious to the spatial distribution of servers and users. The spatial distribution of servers and users in a service network is vital in determining the overall performance of the service. For example, the Euclidean distance between a user request and its allocated server, also known as request distance, directly translates to communication latency incurred by a user when accessing the service. Also, in wireless networks, signal attenuation is strongly coupled to request distance, therefore developing allocation policies to minimize request distance can help reduce energy consumption. Thus the following natural question arises. How do we design a load balancing policy that captures the spatial distribution of users and servers?
In this work, we aim to answer this question. To this end we propose a spatially motivated POT policy: spatial POT (sPOT) in which each user is sequentially allocated to the least loaded server among its two geographically nearest servers. We assume both users and servers are placed in a two-dimensional Euclidean plane. When both servers and users are placed uniformly at random in the Euclidean plane, we map sPOT to a classical balls and bins allocation policy with bins corresponding to the Voronoi regions associated with the second order Voronoi diagram of the set of servers. We show that sPOT performs better than POT in terms of average request distance. However, a lower bound analysis on the asymptotic expected maximum load for sPOT suggests that POT load balancing benefits are not achieved by sPOT. Inspired by the analysis of sPOT, we further propose two assignment policies and empirically show that these policies achieve the best of both request distance and maximum load behavior.
Our contributions are summarized below:
Analysis of sPOT yielding lower bound expressions for asymptotic expected maximum load.
When servers are placed on a two-dimensional grid, we model sPOT using POT on the Delaunay graph associated with the Voronoi tessellation of the set of servers.
When users and servers are placed uniformly at random on a 2-D region, we model sPOT as classical balls and bins allocation policy with bins corresponding to the Voronoi regions associated with the second order Voronoi diagram of the set of servers.
Introduction of two non-uniform server sampling based POT policies to improve load and request distance behavior.
A candidate set based policy that samples nearest servers from a user and applies POT on the candidate set.
A non-uniform distance decaying sampling based POT in which each user samples two servers with probability inversely proportional to square of the distance to the servers.
A simulation study validating the effectiveness of the proposed policies.
The rest of this paper is organized as follows. The next section contains some technical preliminaries. In Section 3 we formally analyze the load behavior of sPOT for different server placement settings. In Section 4, we propose two non-uniform server sampling based POT policies that achieve both better load and request distance behavior. Finally, the conclusion of this work and potential future work are given in Section 5.
2 Technical Preliminaries
We consider a distributed service network where users and servers are located on a two-dimensional Euclidean plane .
2.1 Users and Servers
The users: Users in the service network are denoted as the set with cardinality We assume users are placed on a two-dimensional euclidean plane uniformly at random.
The servers: In a service network, each user is assigned to a server from the server set with cardinality We consider two cases for placing the servers on a two-dimensional euclidean plane, (i) Grid Placement: servers are placed on a square grid topology embedded in Euclidean space (ii) Uniform placement: servers are placed uniformly at random on the euclidean plane.
We consider the case where . We define the following user allocation strategies.
2.2 User Allocation Policies
A user-to-server allocation policy is defined as a mapping such that In this paper, our goal is to analyze the performance of various allocation policies defined in the literature as follows.
Power of One (POO): This policy assigns each user to one of the servers chosen uniformly at random from .
Power of Two (POT): In this policy, sequentially111We assume that the users can communicate among themselves in order to agree on a sequential order and such communication cost between users is negligible. each user samples two servers uniformly at random from and is allocated to the least loaded server.
Spatial Power of One (sPOO): This policy assigns each user to its geographically nearest server.
We also propose new policies to reduce both the maximum load and expected request distance. We define them as follows.
Spatial Power of Two (sPOT): Each user is sequentially allocated to the least loaded server among its two geographically nearest servers.
Candidate set based sPOT (k-sPOT): Each user uniformly at random samples two servers from a candidate set consisting of its geographically nearest servers and is assigned to the leastly loaded server.
Decay based Power of Two (dPOT): This policy sequentially considers each user samples two servers from (without replacement), each with probability proportional to . Here, denotes the euclidean distance between user and server is then assigned to the server with the least load.
2.3 Performance Metric
To evaluate and characterize the performance of various allocation policies, we consider the maximum asymptotic () load across all servers and expected request distance as performance metrics.
Load on a server, , under allocation policy , is defined as
Request distance of a user request under allocation policy , is defined as Here denotes the euclidean distance between a user and a server
2.4 Geometric Structures
We define the following geometric structures that are useful constructs for analyzing various user allocation policies.
2.4.1 Voronoi Diagram
A Voronoi cell around a server is the set of points in that are closer to than to any other server in [bash07]. The Voronoi diagram of is the set of Voronoi cells of servers in .
2.4.2 Delaunay Graph
The Delaunay graph, associated with is defined as follows. Assign the vertex set and add an edge between servers and , i.e. only if the Voronoi cells of and are adjacent.
2.4.3 Higher order Voronoi diagram
A order Voronoi diagram, is defined as partition of into regions such that points in each region have the same closest servers in .
We present a few definitions and basic results associated with majorization theory that we apply to analyze sPOT policy later in Section 3.2.
The vectoris said to majorize the vector (denoted ) if
where is the largest element of .
A function is called Schur-convex if
Consider the following proposition (Chapter 11, [marshall79])
Let be a random variable having the multinomial distribution
be a random variable having the multinomial distribution
where If is a Schur-convex function of , then is a Schur-convex function of .
3 Spatial Power of Two policy
We now analyze the load behavior of sPOT policy for various server placement settings. We assume users are placed uniformly at random on
3.1 sPOT with Grid based server placement
Consider the case where servers are placed on a two dimensional square grid: on with wrap-around. Let be the event that the two nearest servers of are in We prove the following Lemma.
Let denote the Delaunay graph associated with when servers are placed on a two-dimensional square grid. Then
Let denote the event that a random user is closest to among all servers in Denote as the geographically nearest server of Thus we have
It is not difficult to show that all Voronoi cells in have equal areas. As is proportional to the area of the Voronoi cell surrounding we have
Without loss of generality (W.l.o.g.) consider a user placed uniformly at random on as shown in Figure 3. Denote as the triangle associated with vertices and Let We now evaluate Clearly, Area(). We also have
Therefore for are all equal. Let be the set of neighboring servers of a server on the square grid. Thus, we have
We consider the following lemma presented in [kenthapadi06].
Given a -regular graph with nodes representing bins, if balls are thrown into the bins by choosing a random edge and placing into the smaller of the two bins connected by the edge, then the maximum load is at least with high probability of
We now prove the following theorem.
Suppose servers are placed on a two dimensional square grid : on with wrap-around. Let users be placed independently and uniformly at random on Under sPOT, the maximum load over all servers is at least with high probability of
Suppose we map the set of servers to the bins and the users to the balls. The delaunay graph is -regular. Let be an edge in From Lemma 1, it is clear that each user (ball) selects an edge with probability (i.e. uniformly at random) and gets allocated to the smaller of the two servers (bins) connected by under sPOT policy. Thus a direct application of Lemma 2 with proves the theorem. We verify the results in Lemma 1 through simulation for a 2D square grid under sPOT policy as shown in Figure 3. We assign and empirically compute and denote it as edge probability on edge on the Delaunay graph. We also verify the in expression (6) and denote it as vertex probability on the Delaunay graph. It is clear from Figure 3 that the edge probabilities are almost all equal and so are the vertex probabilities.
Note that, Theorem 1 ensures that no POT benefit is observed when servers are placed on a two dimensional square grid.
Note that Theorem 1 applies to other grid graphs such as a triangular grid, i.e. no POT benefit is observed when servers are placed on a two dimensional triangular grid. The delaunay graph corresponding to a triangular grid based server placement is - regular.
3.2 sPOT with Uniform server placement
We now consider the case when both users and servers are placed uniformly at random on We no longer can invoke Lemma 2. This is due to the fact that the Delaunay graph associated with the servers is no longer regular. Also, the edge sampling probabilities are no longer equal. This is evident from our simulation results on the corresponding Delaunay graph as shown in Figure 3. We have servers placed randomly in a 2D square and empirically compute and denote it as edge probability on edge on the Delaunay graph. Note that the edge probabilities, i.e. , are all completely different from each other. Also the Delaunay graph is not regular. Thus we resort to using the second order Voronoi diagram to analyze the maximum asymptotic load behavior.
Consider the second order Voronoi diagram: associated with the set of servers We have the following Lemma [Chapter 3.2, [okabe92]].
The number of Voronoi cells in under uniform server placement is upper bounded by .
We also have the following Lemma.
Consider the following modified version of balls and bin problem. Suppose there are balls and bins. Each ball is thrown into one of the bins according to a probability distribution
bins. Each ball is thrown into one of the bins according to a probability distributionwith being the probability of each ball falling into bin , in an independent manner. Denote to be the random variable associated with the maximum number of balls in any bin. The we have
where is a scalar constant.
Denote as the random variable associated with the load for bin Clearly follows multinomial distribution
We have Clearly, is a schur convex function since and if then . Also, we have (Chapter 1, [marshall79]): whenever with Thus applying Proposition 1 yields
Suppose both users and servers are placed independently and uniformly at random on . Under sPOT policy, the expected maximum load over all servers is at least with high probability of i.e., we do not get POT benefits.
Consider the second order Voronoi diagram: associated with the set of servers W.l.o.g. consider a cell in . The probability that a user selects the server pair as its two nearest servers is proportional to the area of the cell . However, the area distribution of cells in is non-uniform (say with probability distribution ). We can invoke classical balls and bins argument on as follows inspired by the discussion in [kenthapadi06]. We treat each cell in as a bin. Thus by Lemma 3, there are bins (or cells). Each ball (or user) choses a bin (or a cell) from a distribution and let denote the expected maximum asymptotic load across the bins. Let denote the expected maximum asymptotic load across the bins when balls are assigned to bins with each ball choosing a bin uniformly at random. From classical balls and bins theory, Clearly, by Lemma 4, we have Since a cell consists of a server pair, one of the server pair corresponding to the maximum load would have load at least Thus the maximum load across all servers would be at least
3.3 Tradeoff between Load and Request Distance
In this Section, we discuss the inherent tradeoff between maximum load and expected request distance metric among different allocation policies. We evaluate the performance of sPOT as compared to other allocation policies. We consider servers and an equal number of users placed at a unit square uniformly at random. We ran trials for each policy. We compare the performance of various allocation policies in Figure 4.
First, note that with respect to maximum load, the spatial based policies perform worse compared to their classical counterparts. Note that the introduction of spatial aspects into a policy increases its maximum load. For example, sPOT performs worse than both POO and POT as shown in Figure 4 (a). Since the maximum asymptotic load for POO is with high probability, Figure 4 (a) validates our lower bound results obtained for sPOT in Theorem 2.
However, the expected request distance is smallest for sPOO and almost similar to that of sPOT. Also, both POT and POO have very high and similar expected request distances as shown in Figures 4 (b). Both results shown in Figure 4 (a) and (b) combined, illustrate the tradeoff between maximum load and expected request distance metric.
4 Improving Load and Request Distance Behavior
In Section 3, we showed that for both grid and uniform based server placement, sPOT does not provide POT benefits. As POT is oblivious to the spatial aspect of user and server distributions, it performs worse with respect to the expected request distance metric. Thus there exists a tradeoff between maximum load and expected request distance among different allocation policies.
Note that for each user, once its arrival location is fixed, the sampling of two servers in sPOT policy is deterministic while it is completely random for POT. This random sampling over the entire set of servers results in better load behavior for POT than for sPOT. However, since random sampling in POT is oblivious to the distances of servers from the particular user, POT incurs very large expected request distance. Thus if one can design a policy with random and distance dependent sampling of servers, such a policy should provide benefits of both POT and sPOT in terms of maximum load and expected request distance respectively. Below we propose and evaluate two such policies to get benefits of both POT and sPOT. We empirically show that they achieve both POT like load benefits while having a request distance profile similar to that of sPOT.
4.1 Decay based POT (dPOT)
Consider the allocation of a random user in the service network. We propose a decay based POT (dPOT) policy to allocate as follows. Under dPOT, samples two servers from (without replacement), each with probability proportional to . Here, denotes the euclidean distance between user and server then gets allocated to the server with the least load among sampled servers. This rule is similar to the one used in small world routing [Kleinberg00]. Note that, since the sampling probability of a server is inversely proportional to its distance from dPOT incurs a smaller expected request distance compared to POT. Surprisingly, dPOT achieves similar load behavior to that of POT. We compare the performance of dPOT to sPOT and POT as follows.
We perform a single simulation run for each of the policies: dPOT, sPOT, POT and measure the distributions of load values across all the servers and of the request distance. Figure 5 (a) shows the load distribution and Figure 5 (b) shows the request distance distribution for dPOT. We plot the load distribution for POT and request distance distribution for sPOT in Figure 5 (c) and (d) respectively.
First we focus on the server loads in Figure 5 (a) and (c). Interestingly, the load distributions are almost identical for dPOT and POT. Similarly, sPOT performs better than dPOT in terms of request distance distribution as shown in Figure 5 (b) and (d)since they significantly favor closer nodes. However, compared to POT (as shown in Figure 5 (e)), dPOT performs significantly better in terms of request distances. Thus dPOT achieves the best of both worlds, i.e., low maximum load and low distances.
4.2 Candidate set based sPOT (-sPOT)
We now propose a policy that improves the load behavior of sPOT. We define to be the candidate set (of size ) consisting of nearest servers for a particular user. Under -sPOT, the user selects two servers uniformly at random from and assigns itself to the leastly loaded one. Thus, the formation of candidate set with the nearest criteria makes the sampling technique distance dependent. Also, the random sampling of two servers within the candidate set helps to balance load and reduce the overall maximum load. Clearly sPOT and POT are two extremes of the policy -sPOT with and respectively. Below, we discuss the effect of on maximum load and expected request distance behavior and compare it to other policies.
Figure 6 (a) shows the growth of average maximum load (averaged over 50 simulation runs for each point) as is varied from 100 to 40000. We observe that both dPOT and POT perform the best. k-sPOT with performs quite well compared to sPOT. Also, we have observed through simulation that the average maximum load profiles are very similar for k-sPOT with finite k to that of sPOT. Based on these results, we present the following conjecture.
If the candidate set in k-sPOT does not grow with , no POT benefit is expected.
Figure 6 (b) shows how the average request distance drops as increases (since the node density increases). We observe that, not surprisingly, sPOT outperform the rest. However, k-sPOT with performs quite well. Thus k-sPOT with achieves good performance for both load and request distance.
Note that, since dPOT selects servers through distance based sampling, change in positions of servers theoretically requires choosing a new set of sampling distributions. However, sampling in -sPOT depends on the log neighborhood of the user, thus involves less frequent updates for server sampling distributions.
In this work we considered a class of power of two choices based allocation policy where both resources and users are located on a two-dimensional plane. We analyzed the sPOT policy and provided expressions for the lower bound on the asymptotic maximum load on the resources. We claim that for both grid and uniform based resource placement, sPOT does not provide POT benefits. We proposed two non-uniform euclidean distance based server sampling policies that achieved the best load and request distance behavior. Experimental results validate the effectiveness of our proposed policies. Finally, going further, we aim at extending our results to consider dynamic arrival of mobile users on the euclidean plane.