  Bayesian optimization is normally performed within fixed variable bounds. In
cases like hyperparameter tuning for machine learning algorithms, setting the
variable bounds is not trivial. It is hard to guarantee that any fixed bounds
will include the true global optimum. We propose a Bayesian optimization
approach that only needs to specify an initial search space that does not
necessarily include the global optimum, and expands the search space when
necessary. However, over-exploration may occur during the search space
expansion. Our method can adaptively balance exploration and exploitation in an
expanding space. Results on a range of synthetic test functions and an MLP
hyperparameter optimization task show that the proposed method out-performs or
at least as good as the current state-of-the-art methods.
