  Bayesian optimization has emerged as a strong candidate tool for global
optimization of functions with expensive evaluation costs. However, due to the
dynamic nature of research in Bayesian approaches, and the evolution of
computing technology, using Bayesian optimization in a parallel computing
environment remains a challenge for the non-expert. In this report, I review
the state-of-the-art in parallel and scalable Bayesian optimization methods. In
addition, I propose practical ways to avoid a few of the pitfalls of Bayesian
optimization, such as oversampling of edge parameters and over-exploitation of
high performance parameters. Finally, I provide relatively simple, heuristic
algorithms, along with their open source software implementations, that can be
immediately and easily deployed in any computing environment.
