  Many parallel and distributed computing research results are obtained in
simulation, using simulators that mimic real-world executions on some target
system. Each such simulator is configured by picking values for parameters that
define the behavior of the underlying simulation models it implements. The main
concern for a simulator is accuracy: simulated behaviors should be as close as
possible to those observed in the real-world target system. This requires that
values for each of the simulator's parameters be carefully picked, or
"calibrated," based on ground-truth real-world executions. Examining the
current state of the art shows that simulator calibration, at least in the
field of parallel and distributed computing, is often undocumented (and thus
perhaps often not performed) and, when documented, is described as a
labor-intensive, manual process. In this work we evaluate the benefit of
automating simulation calibration using simple algorithms. Specifically, we use
a real-world case study from the field of High Energy Physics and compare
automated calibration to calibration performed by a domain scientist. Our main
finding is that automated calibration is on par with or significantly
outperforms the calibration performed by the domain scientist. Furthermore,
automated calibration makes it straightforward to operate desirable trade-offs
between simulation accuracy and simulation speed.
