  Bayesian Optimization (BO), guided by Gaussian process (GP) surrogates, has
proven to be an invaluable technique for efficient, high-dimensional, black-box
optimization, a critical problem inherent to many applications such as
industrial design and scientific computing. Recent contributions have
introduced reinforcement learning (RL) to improve the optimization performance
on both single function optimization and \textit{few-shot} multi-objective
optimization. However, even few-shot techniques fail to exploit similarities
shared between closely related objectives. In this paper, we combine recent
developments in Deep Kernel Learning (DKL) and attention-based Transformer
models to improve the modeling powers of GP surrogates with meta-learning. We
propose a novel method for improving meta-learning BO surrogates by
incorporating attention mechanisms into DKL, empowering the surrogates to adapt
to contextual information gathered during the BO process. We combine this
Transformer Deep Kernel with a learned acquisition function trained with
continuous Soft Actor-Critic Reinforcement Learning to aid in exploration. This
Reinforced Transformer Deep Kernel (RTDK-BO) approach yields state-of-the-art
results in continuous high-dimensional optimization problems.
