  To optimally deploy their applications, users of Infrastructure-as-a-Service
clouds are required to evaluate the costs and performance of different
combinations of cloud configurations to find out which combination provides the
best service level for their specific application. Unfortunately, benchmarking
cloud services is cumbersome and error-prone. In this paper, we propose an
architecture and concrete implementation of a cloud benchmarking Web service,
which fosters the definition of reusable and representative benchmarks. In
distinction to existing work, our system is based on the notion of
Infrastructure-as-Code, which is a state of the art concept to define IT
infrastructure in a reproducible, well-defined, and testable way. We
demonstrate our system based on an illustrative case study, in which we measure
and compare the disk IO speeds of different instance and storage types in
Amazon EC2.
