  A widely accepted way to assess the performance of iterative black-box
optimizers is to analyze their empirical cumulative distribution function
(ECDF) of pre-defined quality targets achieved not later than a given runtime.
In this work, we consider an alternative approach, based on the empirical
attainment function (EAF) and we show that the target-based ECDF is an
approximation of the EAF. We argue that the EAF has several advantages over the
target-based ECDF. In particular, it does not require defining a priori quality
targets per function, captures performance differences more precisely, and
enables the use of additional summary statistics that enrich the analysis. We
also show that the average area over the convergence curves is a
simpler-to-calculate, but equivalent, measure of anytime performance. To
facilitate the accessibility of the EAF, we integrate a module to compute it
into the IOHanalyzer platform. Finally, we illustrate the use of the EAF via
synthetic examples and via the data available for the BBOB suite.
