  Search algorithms for the bandit problems are applicable in materials
discovery. However, the objectives of the conventional bandit problem are
different from those of materials discovery. The conventional bandit problem
aims to maximize the total rewards, whereas materials discovery aims to achieve
breakthroughs in material properties. The max K-armed bandit (MKB) problem,
which aims to acquire the single best reward, matches with the discovery tasks
better than the conventional bandit. Thus, here, we propose a search algorithm
for materials discovery based on the MKB problem using a pseudo-value of the
upper confidence bound of expected improvement of the best reward. This
approach is pseudo-guaranteed to be asymptotic oracles that do not depends on
the time horizon. In addition, compared with other MKB algorithms, the proposed
algorithm has only one hyperparameter, which is advantageous in materials
discovery. We applied the proposed algorithm to synthetic problems and
molecular-design demonstrations using a Monte Carlo tree search. According to
the results, the proposed algorithm stably outperformed other bandit algorithms
in the late stage of the search process when the optimal arm of the MKB could
not be determined based on its expectation reward.
