Scaling Replicated State Machines with Compartmentalization [Technical Report]

by   Michael Whittaker, et al.

State machine replication protocols, like MultiPaxos and Raft, are a critical component of many distributed systems and databases. However, these protocols offer relatively low throughput due to several bottlenecked components. Numerous existing protocols fix different bottlenecks in isolation but fall short of a complete solution. When you fix one bottleneck, another arises. In this paper, we introduce compartmentalization, the first comprehensive technique to eliminate state machine replication bottlenecks. Compartmentalization involves decoupling individual bottlenecks into distinct components and scaling these components independently. Compartmentalization has two key strengths. First, compartmentalization leads to strong performance. In this paper, we demonstrate how to compartmentalize MultiPaxos to increase its throughput by 6x on a write-only workload and 16x on a mixed read-write workload. Unlike other approaches, we achieve this performance without the need for specialized hardware. Second, compartmentalization is a technique, not a protocol. Industry practitioners can apply compartmentalization to their protocols incrementally without having to adopt a completely new protocol.


Bipartisan Paxos: A Modular State Machine Replication Protocol

There is no shortage of state machine replication protocols. From Genera...

Matchmaker Paxos: A Reconfigurable Consensus Protocol [Technical Report]

State machine replication protocols, like MultiPaxos and Raft, are at th...

Leopard: Scaling BFT without Sacrificing Efficiency

With the emergence of large-scale decentralized applications, a scalable...

Rabia: Simplifying State-Machine Replication Through Randomization

We introduce Rabia, a simple and high performance framework for implemen...

Achieving State Machine Replication without Honesty Assumptions

Current standards for player characterisation in tokenised state machine...

Can 100 Machines Agree?

Agreement protocols have been typically deployed at small scale, e.g., u...

Bernoulli Meets PBFT: Modeling BFT Protocols in the Presence of Dynamic Failures

The publication of the pivotal state machine replication protocol PBFT l...

Please sign up or login with your details

Forgot password? Click here to reset