  Internet-scale services rely on data partitioning and replication to provide
scalable performance and high availability. Moreover, to reduce user-perceived
response times and tolerate disasters (i.e., the failure of a whole
datacenter), services are increasingly becoming geographically distributed.
Data partitioning and replication, combined with local and geographical
distribution, introduce daunting challenges, including the need to carefully
order requests among replicas and partitions. One way to tackle this problem is
to use group communication primitives that encapsulate order requirements. This
paper presents a detailed performance evaluation of Multi-Ring Paxos, a
scalable group communication primitive. We focus our analysis on "extreme
conditions" with deployments including high-end 10 Gbps networks, a large
number of combined rings (i.e., independent Paxos instances), a large number of
replicas in a ring, and a global deployment. We also report on the performance
of recovery under peak load and present two novel extensions to boost
Multi-Ring Paxos's performance.
