  Introducing new algorithmic ideas is a key part of the continuous improvement
of existing optimization algorithms. However, when introducing a new component
into an existing algorithm, assessing its potential benefits is a challenging
task. Often, the component is added to a default implementation of the
underlying algorithm and compared against a limited set of other variants. This
assessment ignores any potential interplay with other algorithmic ideas that
share the same base algorithm, which is critical in understanding the exact
contributions being made. We introduce a more extensive procedure, which uses
hyperparameter tuning as a means of assessing the benefits of new algorithmic
components. This allows for a more robust analysis by not only focusing on the
impact on performance, but also by investigating how this performance is
achieved. We implement our suggestion in the context of the Modular CMA-ES
framework, which was redesigned and extended to include some new modules and
several new options for existing modules, mostly focused on the step-size
adaptation method. Our analysis highlights the differences between these new
modules, and identifies the situations in which they have the largest
contribution.
