  Probabilistic programming systems enable users to encode model structure and
naturally reason about uncertainties, which can be leveraged towards improved
Bayesian optimization (BO) methods. Here we present a probabilistic program
embedding of BO that is capable of addressing main issues such as problematic
domains (noisy, non-smooth, high-dimensional) and the neglected
inner-optimization. Not only can we utilize programmable structure to
incorporate domain knowledge to aid optimization, but dealing with
uncertainties and implementing advanced BO techniques become trivial, crucial
for use in practice (particularly for non-experts). We demonstrate the efficacy
of the approach on optimization benchmarks and a real-world drug development
scenario.
