  Bayesian optimization (BO) is a well-established method to optimize black-box
functions whose direct evaluations are costly. In this paper, we tackle the
problem of incorporating expert knowledge into BO, with the goal of further
accelerating the optimization, which has received very little attention so far.
We design a multi-task learning architecture for this task, with the goal of
jointly eliciting the expert knowledge and minimizing the objective function.
In particular, this allows for the expert knowledge to be transferred into the
BO task. We introduce a specific architecture based on Siamese neural networks
to handle the knowledge elicitation from pairwise queries. Experiments on
various benchmark functions with both simulated and actual human experts show
that the proposed method significantly speeds up BO even when the expert
knowledge is biased compared to the objective function.
