  Recent secure aggregation protocols enable privacy-preserving federated
learning for high-dimensional models among thousands or even millions of
participants. Due to the scale of these use cases, however, end-to-end
empirical evaluation of these protocols is impossible. We present OLYMPIA, a
framework for empirical evaluation of secure protocols via simulation. OLYMPIA.
provides an embedded domain-specific language for defining protocols, and a
simulation framework for evaluating their performance. We implement several
recent secure aggregation protocols using OLYMPIA, and perform the first
empirical comparison of their end-to-end running times. We release OLYMPIA as
open source.
