CARE: Resource Allocation Using Sparse Communication
share
We propose a new framework for studying effective resource allocation in a load balancing system under sparse communication, a problem that arises, for instance, in data centers. At the core of our approach is state approximation, where the load balancer first estimates the servers' states via a carefully designed communication protocol, and subsequently feeds the said approximated state into a load balancing algorithm to generate a routing decision. Specifically, we show that by using a novel approximation algorithm and server-side-adaptive communication protocol, the load balancer can obtain good queue-length approximations using a communication frequency that decays quadratically in the maximum approximation error. Furthermore, using a diffusion-scaled analysis, we prove that the load balancer achieves asymptotically optimal performance whenever the approximation error scales at a lower rate than the square-root of the total processing capacity, which includes as a special case constant-error approximations. Using simulations, we find that the proposed policies achieve performance that matches or outperforms the state-of-the-art load balancing algorithms while reducing communication rates by as much as 90 possible to achieve good performance even under very sparse communication, and provide strong evidence that approximate states serve as a robust and powerful information intermediary for designing communication-efficient load balancing systems.
READ FULL TEXT