  We formulate selecting the best optimizing system (SBOS) problems and provide
solutions for those problems. In an SBOS problem, a finite number of systems
are contenders. Inside each system, a continuous decision variable affects the
system's expected performance. An SBOS problem compares different systems based
on their expected performances under their own optimally chosen decision to
select the best, without advance knowledge of expected performances of the
systems nor the optimizing decision inside each system. We design
easy-to-implement algorithms that adaptively chooses a system and a choice of
decision to evaluate the noisy system performance, sequentially eliminates
inferior systems, and eventually recommends a system as the best after spending
a user-specified budget. The proposed algorithms integrate the stochastic
gradient descent method and the sequential elimination method to simultaneously
exploit the structure inside each system and make comparisons across systems.
For the proposed algorithms, we prove exponential rates of convergence to zero
for the probability of false selection, as the budget grows to infinity. We
conduct three numerical examples that represent three practical cases of SBOS
problems. Our proposed algorithms demonstrate consistent and stronger
performances in terms of the probability of false selection over benchmark
algorithms under a range of problem settings and sampling budgets.
