  Accurate uncertainty estimates can significantly improve the performance of
iterative design of experiments, as in Sequential and Reinforcement learning.
For many such problems in engineering and the physical sciences, the design
task depends on multiple correlated model outputs as objectives and/or
constraints. To better solve these problems, we propose a recalibrated
bootstrap method to generate multivariate prediction intervals for bagged
models and show that it is well-calibrated. We apply the recalibrated bootstrap
to a simulated sequential learning problem with multiple objectives and show
that it leads to a marked decrease in the number of iterations required to find
a satisfactory candidate. This indicates that the recalibrated bootstrap could
be a valuable tool for practitioners using machine learning to optimize systems
with multiple competing targets.
