  The key to Black-Box Optimization is to efficiently search through input
regions with potentially widely-varying numerical properties, to achieve
low-regret descent and fast progress toward the optima. Monte Carlo Tree Search
(MCTS) methods have recently been introduced to improve Bayesian optimization
by computing better partitioning of the search space that balances exploration
and exploitation. Extending this promising framework, we study how to further
integrate sample-based descent for faster optimization. We design novel ways of
expanding Monte Carlo search trees, with new descent methods at vertices that
incorporate stochastic search and Gaussian Processes. We propose the
corresponding rules for balancing progress and uncertainty, branch selection,
tree expansion, and backpropagation. The designed search process puts more
emphasis on sampling for faster descent and uses localized Gaussian Processes
as auxiliary metrics for both exploitation and exploration. We show empirically
that the proposed algorithms can outperform state-of-the-art methods on many
challenging benchmark problems.
