  Many optimization problems admit a number of local optima, among which there
is the global optimum. For these problems, various heuristic optimization
methods have been proposed. Comparing the results of these solvers requires the
definition of suitable metrics. In the electrical energy systems literature,
simple metrics such as best value obtained, the mean value, the median or the
standard deviation of the solutions are still used. However, the comparisons
carried out with these metrics are rather weak, and on these bases a somehow
uncontrolled proliferation of heuristic solvers is taking place. This paper
addresses the overall issue of understanding the reasons of this proliferation,
showing a conceptual scheme that indicates how the assessment of the best
solver may result in the unlimited formulation of new solvers. Moreover, this
paper shows how the use of more refined metrics defined to compare the
optimization result, associated with the definition of appropriate benchmarks,
may make the comparisons among the solvers more robust. The proposed metrics
are based on the concept of first-order stochastic dominance and are defined
for the cases in which: (i) the globally optimal solution can be found (for
testing purposes); and (ii) the number of possible solutions is so large that
practically it cannot be guaranteed that the global optimum has been found.
Illustrative examples are provided for a typical problem in the electrical
energy systems area-distribution network reconfiguration. The conceptual
results obtained are generally valid to compare the results of other
optimization problems.
