  The goal of Multi-task Bayesian Optimization (MBO) is to minimize the number
of queries required to accurately optimize a target black-box function, given
access to offline evaluations of other auxiliary functions. When offline
datasets are large, the scalability of prior approaches comes at the expense of
expressivity and inference quality. We propose JUMBO, an MBO algorithm that
sidesteps these limitations by querying additional data based on a combination
of acquisition signals derived from training two Gaussian Processes (GP): a
cold-GP operating directly in the input domain and a warm-GP that operates in
the feature space of a deep neural network pretrained using the offline data.
Such a decomposition can dynamically control the reliability of information
derived from the online and offline data and the use of pretrained neural
networks permits scalability to large offline datasets. Theoretically, we
derive regret bounds for JUMBO and show that it achieves no-regret under
conditions analogous to GP-UCB (Srinivas et. al. 2010). Empirically, we
demonstrate significant performance improvements over existing approaches on
two real-world optimization problems: hyper-parameter optimization and
automated circuit design.
