  Feature-based offline algorithm selection has shown its effectiveness in a
wide range of optimization problems, including the black-box optimization
problem. An algorithm selection system selects the most promising optimizer
from an algorithm portfolio, which is a set of pre-defined optimizers. Thus,
algorithm selection requires a well-constructed algorithm portfolio consisting
of efficient optimizers complementary to each other. Although construction
methods for the fixed-target setting have been well studied, those for the
fixed-budget setting have received less attention. Here, the fixed-budget
setting is generally used for computationally expensive optimization, where a
budget of function evaluations is small. In this context, first, this paper
points out some undesirable properties of experimental setups in previous
studies. Then, this paper argues the importance of considering the number of
function evaluations used in the sampling phase when constructing algorithm
portfolios, whereas the previous studies ignored that. The results show that
algorithm portfolios constructed by our approach perform significantly better
than those by the previous approach.
