  Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find
optimal sampling policies through solving a dynamic program (DP) that maximizes
a long-term reward over a rolling horizon. Though promising, lookahead BO faces
the risk of error propagation through its increased dependence on a possibly
mis-specified model. In this work we focus on the rollout approximation for
solving the intractable DP. We first prove the improving nature of rollout in
tackling lookahead BO and provide a sufficient condition for the used heuristic
to be rollout improving. We then provide both a theoretical and practical
guideline to decide on the rolling horizon stagewise. This guideline is built
on quantifying the negative effect of a mis-specified model. To illustrate our
idea, we provide case studies on both single and multi-information source BO.
Empirical results show the advantageous properties of our method over several
myopic and non-myopic BO algorithms.
