With the advent of cloud computing and emergence of big data, scale-out data caching systems are widely deployed and their horizontal scalability  becomes increasingly important. As an effective solution, consistent hashing  has been commonly used by key-value caching systems, e.g., Dynamo , Aerospike , Memcached , Redis . Using consistent hashing, a large number of servers are organized together to form a cooperative cluster. These servers jointly provide a unified but distributed hash table to serve swift and voluminous data item requests. Once a data item is hashed to one of the hosting servers, most key-value caching systems use the least-recently-used (LRU) caching algorithm, or its variations, e.g., LRU Clock , to decide which data items should be kept in its own individual cache space. These data caching systems play a critical role in optimizing the way information is delivered in Web services.
With consistent hashing, the total amount of cache spaces in the cluster can be easily expanded (scaled horizontally) through the addition of new cache servers. However, these individual cache spaces on different servers are not simply added together to achieve an overall request miss ratio. Although LRU caching on a single server has already been extensively studied, theoretically characterizing the miss ratio of a LRU cluster organized by consistent hashing still remains an unexplored problem. One difficulty in analysis is that the data item requests are shuffled to a large number of servers according to a random hash function. This random hashing complicates the system behavior. Due to the fundamental role and predominant usage in practice, LRU caching with consistent hashing merits a deep investigation.
Characterizing the cache miss behavior of a cluster in such a complex setting not only helps resource planning but also improves the way a cache cluster is organized. To this end, we derive the asymptotic miss ratio of a LRU caching cluster with consistent hashing under the independent reference model (IRM) 
. Interestingly, these individual cache spaces on different servers, though isolated physically but logically connected through a hash function, can be effectively viewed as if they could be pooled together to form a single virtual LRU cache space. Interestingly, this virtual LRU cache has a equivalent cache size determined by the distribution of the random hash function. This equivalence can be established when the cache sizes of the individual servers are large. Our result provides a convenient abstraction that can rigorously relate the more complex LRU caching with consistent hashing to the relatively simpler single LRU cache. Based on this abstraction, many known results on a single LRU cache can be directly translated to a LRU cluster with consistent hashing. Specifically, we prove a characteristic time approximation, previously established for a single LRU cache, for a LRU cluster. Notably, the characteristic time approximation has the same form for almost all of the random hash functions. This result is not straightforward in view that the miss ratio of each of the server is a random variable conditional on the random hash function. However, the overall conditional asymptotic miss ratio of the cluster is always the same almost surely, depending on the probability distribution of the random hash function. Due to this equivalence, we comment that the engineering implications discussed in for a single LRU cache can also be extended to a LRU cluster.
To put the analysis on a concrete basis, we first summarize the important features of consistent hashing and LRU caching.
I-A1 Consistent hashing
Data items are usually organized in a key value pair, and the entire data are stored in the whole cluster as a distributed hash table according to keys.
To scale out the system horizontally, each server is maintained independent, e.g., by using consistent hashing  to select a unique server for each key. The basic idea is to first hash the data items to a large number of partitions at random. These partitions form a ring, as illustrated in Fig. 1. The physical servers, with a total number that is much smaller than the number of partitions, are also hashed to a subset of the partitions. A data item, after being hashed to a partition, will be stored on the server that is closest in the clockwise direction to its associated partition. For example, partitions A, B and C are all stored on server in Fig.1. Therefore, when the server locations are fixed, each server hosts a certain number of partitions that are determined by the hash function.
I-A2 LRU caching
Each data item is hashed to one of the hosting servers. A cache replacement algorithm is needed to manage the cache space on an individual server. Due to the low cost of tracking access history, LRU caching algorithm has been widely used, e.g., for Memcached [5, 10]. A data item is added to the cache after a client has requested it and failed. When the cache is full, the LRU caching algorithm evicts a data item that has not been used for the longest time in order to accommodate the newly requested one.
I-B Contributions of this paper
(I) For a family of hash functions under the Simple Uniform Hashing Assumption (SUHA) , we characterize the asymptotic miss ratio of data item requests on a cluster with consistent hashing. This asymptotic result, expressed as a conditional probability, holds almost surely for all random hash functions in . It provides a new analytical framework to study LRU caching with consistent hashing by conditioning on the random hash function, which is an interesting feature that most existing asymptotic results do not have.
(II) We rigorously establish a one-to-one equivalence between a cluster with consistent hashing and a single virtual LRU cache space with a proper size parametrized by the distribution of the random hash function. Conveniently, this equivalence translates the analytic results from the well-studied single server to the complex cluster with consistent hashing. Based on this equivalence, we prove the characteristic time approximation for a cluster to characterize the miss probability.
(III) Extensive simulations show that our asymptotic results match with the empirical results accurately even for relatively small cache sizes.
I-C Related work
Consistent hashing has gained much popularity in recent years due to the increasing demand of processing large data sets on a scale-out infrastructure. It has been successfully used in a number of real-world applications, e.g., web caches [12, 13], peer-to-peer networks  and distributed storage systems . Most theoretical studies on consistent hashing focus on characterizing the randomized partitioning algorithms to balance data allocation , memory sharing  and perfect hashing . These algorithms are usually analyzed under SUHA. Some works circumvent this assumption using realistic hash functions through simulations [19, 20]. However, none of these works investigate the miss probabilities of a cluster with consistent hashing.
There is a large body of work on the miss ratio of a single LRU server. Different methods have been proposed, e.g., approximation by iterative algorithms , mean field analysis  and the characteristic time approximation [23, 24]. To obtain insights, asymptotic results for Zipf’s popularity distributions have been derived [25, 26, 27]. The characteristic time approximation is also a common approach, which has been shown to be accurate in practical applications [28, 29]. Its success has been supported by the analysis [30, 31]. Nevertheless, these results cannot be directly extended to a cluster with consistent hashing.
Although single LRU caches have been studied in depth, characterizing the miss behavior of cache networks with general topologies remains difficult. Instead, some existing works, e.g., [32, 33], focus on offline optimization problems (e.g., content placement) in cache networks. Some specially structured cache networks (e.g., tree or line networks) have been studied [34, 35] using a TTL-based eviction scheme. With consistent hashing, the cache network can be viewed as a one-hop network, which is the focus of this paper.
Ii Model description
Consider a cluster with servers organized by consistent hashing. Assume that the data item requests hosted on can access an infinite number of distinct data items of unit size that are represented by a sequence . The notation represents that a data item , together with all requests that ask for , are hashed to server by the hash function . Thus, a subsequence of data items, denoted by , are hashed to server , selected from .
To characterize the hash function , we assume the Simple Uniform Hashing Assumption (SUHA) to facilitate the analysis. In practice, the ideal SUHA property is not feasible, and people resort to a (strongly) universal hash family  or a -independent hash family  as approximations. Actually, it has been shown that 2-independent hash functions under mild conditions approximate truly random functions . Specifically, we consider a family of hash functions . Assume that, with chosen from uniformly at random, each data item is dispatched also uniformly at random to one of the partitions. Note that although is random, it becomes one of the deterministic hash function once the random selection is completed. Using consistent hashing, the involved servers store the data items from mutually exclusive subsets of the partitions, as shown in Fig. 1. Since the number of partitions assigned to each of the servers could be different, we can equivalently assume that each data item is independently hashed to server with probability by . In other words, we have the following assumption.
(SUHA) are independent random variables with .
Assume that the arrivals of the data item requests occur at time points . Let be the index of the server for the request at time . The event represents that the request at time is processed on server . Denote by the requested data item at time . Thus, the event means that the request at time is to fetch data item on server . In order to compute the miss ratio when the system reaches stationarity, we consider the request at time . It has been shown  that the miss ratio is equal to the probability that the data item requested by is not in the cache. For cluster , define
Note that is a random sequence determined by the random hash function . We assume that the data items are sorted such that the sequence is non-increasing with respect to . Since is a random subsequence of , is also non-increasing by this ordering. Let , which represents the subsequence of that is hashed to server by . Therefore, . For notational convenience in our proofs, we define . On server , we have . We emphasize that is a random variable determined by the random hash function , which normalizes to be a legitimate distribution.
The data item popularity is assumed to follow a Zipf’s distribution . This is a typical distribution that has been empirically observed in web pages , content-centric network , and video systems . To simplify the analysis, this paper only considers a Zipf’s distribution with . For , we can conduct a similar analysis based on existing results [42, 43, 44].
LRU is equivalent to the move-to-front (MTF) policy [26, 45], which sorts the data items in an increasing order of their last access time. When a data item is requested under MTF, it is moved to the first position of the list and all the other data items that were in front of this one increase their positions by one. Define to be the position of the data item requested by in the sorted list under MTF on the server that processes the request . Then, the miss probability of the requests on server with a cache size is given by , which is conditional on the random hash function and the event that occurs on server , i.e., . Combining the miss ratios of the servers, we obtain the overall miss probability of the cluster , conditional on ,
In the analysis, we assume for .
Iii Main results
In this section, we first derive the miss ratio for each of the servers of the LRU cluster with consistent hashing conditional on the random hash function. Then, we show that these individual cache servers can be regarded as a single virtual LRU server with a proper cache size. This connection also proves the characteristic time approximation for a cluster.
Iii-a Asymptotic miss ratio under random hashing
We derive the miss probabilities for the servers of the cluster by conditioning on the random hash function . Note that uniquely determines . The gamma function is given by . The notation means .
Under the assumptions in Section II, we obtain, for all , almost surely for all , as ,
which implies, almost surely for all ,
The proof is presented in Section VI-A. ∎
This asymptotic result in (4) involves random variables that are determined by . Interestingly, the overall asymptotic miss ratio of the whole cluster in (5) is independent of since . These asymptotic results hold a.s. for all . See Experiments in Section IV. If there is only one server in the cluster , i.e., , Theorem 1 reproduces the results in [25, 26] for a Zipf’s distribution, e.g., Theorem 3 of  on an asymptotic miss probability of a single LRU server. However, extending this result from a single server to a cluster is complicated. We discuss two main issues that cause the difficulty: 1) Theorem 3 of  assumes a deterministic popularity distribution on a server. This condition is not satisfied in our model due to the random hash function; 2) the proofs of [25, 26] cannot be used to prove the characteristic time approximation for a cluster. Because of these reasons, we use a different approach to derive the miss probability of a LRU cluster with consistent hashing, which also proves the characteristic time approximation for a cluster.
Now, suppose that we have a single virtual LRU cache server of size that serves the entire data item requests , which at the same time are also served on the cluster . Based on Theorem 1, we establish an equivalence between the cluster and the virtual LRU cache. Denote by the miss probability of the virtual LRU cache conditional on . Recall that the server has a cache capacity .
Under the assumptions in Section II, we obtain, almost surely for all ,
The proof is presented in Section VI-B. ∎
This theorem shows that the miss probability on the cluster is asymptotically equal to the miss ratio of a LRU server with the cache size given by (7). Interestingly, as illustrated in Experiment 2, this asymptotic equivalence is accurate even when the cache size of each individual server of cluster is relatively small. Using this connection, existing results and insights that have been established for a single server seem to be also true for a LRU cluster with consistent hashing. This could be useful for resource planning and cluster optimization.
Iii-B Characteristic time approximation with consistent hashing
The characteristic time approximation 
has been widely used in estimating the miss ratio of a LRU server. Based on the connection between a cluster and a single virtual LRU server established by Theorem2, we derive the characteristic time approximation for a cluster. Recall that the server has a cache capacity .
Theorem 1 shows that, although the miss ratio of each server is random, determined by , the overall asymptotic miss ratio of the cluster is independent of . This interesting result motivates us to define the characteristic time approximation for the cluster
Under the assumptions of Theorem 2, we have, almost surely for all ,
The proof is presented in Section VI-C. ∎
In this section, we conduct extensive simulations using C++ to verify the main results in Section III. Notably, all simulations match with our theoretical results even for relatively small cache sizes.
Experiment 1. This experiment verifies Theorem 1. Consider a cluster of heterogeneous servers that have distinct cache sizes and different hashing probabilities. The server has a cache capacity with selected uniformly at random from . Thus, is the average cache size across all of the servers. Recall that is the probability that a data item is hashed to server .
Let for . Conditional on , we obtain random variables with . Set the total number of data items , and the popularity distribution with . For each , we first simulate requests to ensure that the entire system reaches stationary, and then more requests to compute the empirical miss probabilities of the cluster and the individual servers. To verify (4), we need to show that it holds for all . To visualize the results, we only plot the miss probabilities of five servers in Fig. 2. The empirical results match well with the the theoretical results by (4) and (5) even when is small.
We demonstrate the accuracy of the characteristic time approximationin (8), which also verifies Theorem 3. By computing (7), we obtain the equivalent size of the virtual LRU caching space . For the characteristic time approximation of the cluster , we use a binary search to find the solution of the equation and then calculate the miss probability by (8). It can be shown from the left figure in Fig. 3 that the empirical results match well with the theoretical results for the miss probabilities of the virtual LRU caching server and the cluster even for . In addition, the characteristic time approximation (8) provides an accurate estimation of the miss ratio of the cluster .
Experiment 3. This experiment moves beyond the assumptions of this paper and considers a realistic setting. Thus, we cannot explicitly compute the equivalent virtual cache size by Theorem 2. However, we still demonstrate an equivalence between the cluster and a virtual cache. We set and use a 2-independent hash function . For a cluster of servers described in Experiment IV, we hash each server to one of partitions using the 2-independent hash function , where are chosen from uniformly at random with a large prime . Using the same hash function, each data item is hashed to one of these partitions. The data items from partition are stored on the server that has an index if the set and otherwise.
We set the total number of data items , and the popularity distribution with . For the virtual LRU caching space, we find the equivalent size . It can be shown from Fig. 4 that the empirical miss ratios of the cluster and the virtual LRU cache match very well.
Driven by the trend to scale out caching systems for processing big data, LRU caching with consistent hashing has been widely deployed. We develop a theoretical framework to investigate the miss ratio of a LRU cluster for a family of hash functions satisfying the Simple Uniform Hashing Assumption (SUHA). We derive a close-form asymptotic miss probability that holds almost surely for all of the random hash functions from this family. This result also establishs a one-to-one equivalence between a LRU cluster and a single virtual LRU server. It provides a convenient abstraction to understand the complex LRU cluster using the insights obtained from a LRU server. Based on this connection, we also prove the characteristic time approximation for a cluster with consistent hashing.
This section contains the proofs of our main theorems.
Vi-a Proof of Theorem 1
We rely on the following Lemma 1 to prove Theorem 1. To this end, we use Lemmas 5, 6 and 7 to study the three quantities in Lemma 1, i.e., , and . Specifically, we first note that the variables can be regarded as the indices of data items on server and show that holds with high probability in Lemma 5. Then, we prove that the functional relationship between and satisfies with high probability in Lemma 6. Last, we show with high probability in Lemma 7. Using and Borel-Cantelli lemma, we prove that Theorem 1 holds almost surely for all .
Define an increasing function with an inverse . We say that as if ; has a complementary definition.
If , and , then, as ,
Let be the largest integer such that . Since the requests are i.i.d, then we obtain , which, in conjunction with , indicates that
In the following step, we show that . For simple notions, we define an increasing and continuous function . Noting and using (10), we obtain, for , there exists such that, for
where is the inverse of . Using in (12), we obtain
For any and a sufficiently large n, consider a sequence of indices such that for . Then, we have
where the second inequality follows from the fact that for . Letting , we obtain
where the second inequality follows from for . Based on Theorem 1.2.1 of , we have, if ,
which, using and passing and , implies that
Since , then for any , there exists such that for , , which, in conjunction with (VI-A) and , yields
Replacing by in (VI-A) and letting , we obtain
Next, we use (18) to prove (11). Define as the number of the different data items that have been requested at time and let the inverse of be . It is not hard to show the event is equivalent to . Thus, we have
Define Bernoulli random variables and let imply that the data item has been requested at time and otherwise. Let . Noting , we obtain . For any , using Markov inequality and , we have
When is small enough, we have . Thus, using (VI-A), we obtain
Letting in (21) yields
Let . Then, by (22), we obtain
Noting and passing , we have, . Thus, passing and using (VI-A), we obatin
Letting , by a similar approach to (25), we obtain, as ,
We introduce some necessary definitions. Define mutually independent Bernoulli random variables . Let indicate that the data item is hashed to server and otherwise. We have . Define , which represents the number of data items hashed to server from . Let and . We quote Bernstein’s inequality in Lemma 2, and establish the following Lemma 3 to estimate , which will be used to estimate in (44).
Lemma 2 (Theorem in ).
For independent random variables with , we obtain, ,
There exist and such that for ,