  State machine replication protocols, like MultiPaxos and Raft, are a critical
component of many distributed systems and databases. However, these protocols
offer relatively low throughput due to several bottlenecked components.
Numerous existing protocols fix different bottlenecks in isolation but fall
short of a complete solution. When you fix one bottleneck, another arises. In
this paper, we introduce compartmentalization, the first comprehensive
technique to eliminate state machine replication bottlenecks.
Compartmentalization involves decoupling individual bottlenecks into distinct
components and scaling these components independently. Compartmentalization has
two key strengths. First, compartmentalization leads to strong performance. In
this paper, we demonstrate how to compartmentalize MultiPaxos to increase its
throughput by 6x on a write-only workload and 16x on a mixed read-write
workload. Unlike other approaches, we achieve this performance without the need
for specialized hardware. Second, compartmentalization is a technique, not a
protocol. Industry practitioners can apply compartmentalization to their
protocols incrementally without having to adopt a completely new protocol.
