  Peak performance metrics published by vendors often do not correspond to what
can be achieved in practice. It is therefore of great interest to do extensive
benchmarking on core applications and library routines. Since DGEMM is one of
the most used in compute-intensive numerical codes, it is typically highly
vendor optimized and of great interest for empirical benchmarks. In this paper
we show how to build a novel tool that autotunes the benchmarking process for
the Roofline model. Our novel approach can efficiently and reliably find
optimal configurations for any target hardware. Results of our tool on a range
of hardware architectures and comparisons to theoretical peak performance are
included.
  Our tool autotunes the benchmarks for the target architecture by deciding the
optimal parameters through state space reductions and exhaustive search. Our
core idea includes calculating the confidence interval using the variance and
mean and comparing it against the current optimum solution. We can then
terminate the evaluation process early if the confidence interval's maximum is
lower than the current optimum solution. This dynamic approach yields a search
time improvement of up to 116.33x for the DGEMM benchmarking process compared
to a traditional fixed sample-size methodology. Our tool produces the same
benchmarking result with an error of less than 2% for each of the optimization
techniques we apply, while providing a great reduction in search time. We
compare these results against hand-tuned benchmarking parameters. Results from
the memory-intensive TRIAD benchmark, and some ideas for future directions are
also included.
