LB Scalability: Achieving the Right Balance Between Being Stateful and Stateless

10/26/2020
by   Reuven Cohen, et al.
0

A high performance Layer-4 load balancer (LB) is one of the most important components of a cloud service infrastructure. Such an LB uses network and transport layer information for deciding how to distribute client requests across a group of servers. A crucial requirement for a stateful LB is per connection consistency (PCC); namely, that all the packets of the same connection will be forwarded to the same server, as long as the server is alive, even if the pool of servers or the assignment function changes. The challenge is in designing a high throughput, low latency solution that is also scalable. This paper proposes a highly scalable LB, called Prism, implemented using a programmable switch ASIC. As far as we know, Prism is the first reported LB that can process millions of connections per second and hundreds of millions connections in total, while ensuring PCC. This is due to the fact that Prism forwards all the packets in hardware, even during server pool changes, while avoiding the need to maintain a hardware state per every active connection. We implemented a prototype of the proposed architecture and showed that Prism can scale to 100 million simultaneous connections, and can accommodate more than one pool update per second.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/25/2023

NetClone: Fast, Scalable, and Dynamic Request Cloning for Microsecond-Scale RPCs

Spawning duplicate requests, called cloning, is a powerful technique to ...
research
10/12/2020

RackSched: A Microsecond-Scale Scheduler for Rack-Scale Computers (Technical Report)

Low-latency online services have strict Service Level Objectives (SLOs) ...
research
10/27/2021

Charon: Load-Aware Load-Balancing in P4

Load-Balancers play an important role in data centers as they distribute...
research
12/07/2020

Machine Learning Prediction of Gamer's Private Networks

The Gamer's Private Network (GPN) is a client/server technology created ...
research
11/16/2021

HyperNAT: Scaling Up Network AddressTranslation with SmartNICs for Clouds

Network address translation (NAT) is a basic functionality in cloud gate...
research
02/06/2018

RDMAvisor: Toward Deploying Scalable and Simple RDMA as a Service in Datacenters

RDMA is increasingly adopted by cloud computing platforms to provide low...
research
09/19/2023

Evaluating the Benefits: Quantifying the Effects of TCP Options, QUIC, and CDNs on Throughput

To keep up with increasing demands on quality of experience, assessing a...

Please sign up or login with your details

Forgot password? Click here to reset