  Reliable distributed systems require replication and consensus among
distributed processes to tolerate process and communication failures.
Understanding and assuring the correctness of protocols for replication and
consensus have been a significant challenge. This paper describes the precise
specification and runtime checking of Derecho, a more recent, sophisticated
protocol for fast replication and consensus for cloud services.
  A precise specification must fill in missing details and resolve ambiguities
in English and pseudocode algorithm descriptions while also faithfully
following the descriptions. To help check the correctness of the protocol, we
also performed careful manual analysis and increasingly systematic runtime
checking. We obtain a complete specification that is directly executable, and
we discover and fix a number of issues in the pseudocode. These results were
facilitated by the already detailed pseudocode of Derecho and made possible by
using DistAlgo, a language that allows distributed algorithms to be easily and
clearly expressed and directly executed.
