  Catalyzed by the popularity of blockchain technology, there has recently been
a renewed interest in the design, implementation and evaluation of
decentralized systems. Most of these systems are intended to be deployed at
scale and in heterogeneous environments with real users and unpredictable
workloads. Nevertheless, most research in this field evaluates such systems in
controlled environments that poorly reflect the complex conditions of
real-world environments. In this work, we argue that deployment is crucial to
understanding decentralized mechanisms in a real-world environment and an
enabler to building more robust and sustainable systems. We highlight the
merits of deployment by comparing this approach with other experimental setups
and show how our lab applied a deployment-first methodology. We then outline
how we use Tribler, our peer-to-peer file-sharing application, to deploy and
monitor decentralized mechanisms at scale. We illustrate the application of our
methodology by describing a deployment trial in experimental tokenomics.
Finally, we summarize four lessons learned from multiple deployment trials
where we applied our methodology.
