  Gaussian process optimization is a successful class of algorithms(e.g.
GP-UCB) to optimize a black-box function through sequential evaluations.
However, for functions with continuous domains, Gaussian process optimization
has to rely on either a fixed discretization of the space, or the solution of a
non-convex optimization subproblem at each evaluation. The first approach can
negatively affect performance, while the second approach requires a heavy
computational burden. A third option, only recently theoretically studied, is
to adaptively discretize the function domain. Even though this approach avoids
the extra non-convex optimization costs, the overall computational complexity
is still prohibitive. An algorithm such as GP-UCB has a runtime of $O(T^4)$,
where $T$ is the number of iterations. In this paper, we introduce Ada-BKB
(Adaptive Budgeted Kernelized Bandit), a no-regret Gaussian process
optimization algorithm for functions on continuous domains, that provably runs
in $O(T^2 d_\text{eff}^2)$, where $d_\text{eff}$ is the effective dimension of
the explored space, and which is typically much smaller than $T$. We
corroborate our theoretical findings with experiments on synthetic non-convex
functions and on the real-world problem of hyper-parameter optimization,
confirming the good practical performances of the proposed approach.
