  Bayesian optimization is a principled optimization strategy for a black-box
objective function. It shows its effectiveness in a wide variety of real-world
applications such as scientific discovery and experimental design. In general,
the performance of Bayesian optimization is reported through regret-based
metrics such as instantaneous, simple, and cumulative regrets. These metrics
only rely on function evaluations, so that they do not consider geometric
relationships between query points and global solutions, or query points
themselves. Notably, they cannot discriminate if multiple global solutions are
successfully found. Moreover, they do not evaluate Bayesian optimization's
abilities to exploit and explore a search space given. To tackle these issues,
we propose four new geometric metrics, i.e., precision, recall, average degree,
and average distance. These metrics allow us to compare Bayesian optimization
algorithms considering the geometry of both query points and global optima, or
query points. However, they are accompanied by an extra parameter, which needs
to be carefully determined. We therefore devise the parameter-free forms of the
respective metrics by integrating out the additional parameter. Finally, we
empirically validate that our proposed metrics can provide more delicate
interpretation of Bayesian optimization algorithms, on top of assessment via
the conventional metrics.
