  Active learning parallelization is widely used, but typically relies on
fixing the batch size throughout experimentation. This fixed approach is
inefficient because of a dynamic trade-off between cost and speed -- larger
batches are more costly, smaller batches lead to slower wall-clock run-times --
and the trade-off may change over the run (larger batches are often preferable
earlier). To address this trade-off, we propose a novel Probabilistic Numerics
framework that adaptively changes batch sizes. By framing batch selection as a
quadrature task, our integration-error-aware algorithm facilitates the
automatic tuning of batch sizes to meet predefined quadrature precision
objectives, akin to how typical optimizers terminate based on convergence
thresholds. This approach obviates the necessity for exhaustive searches across
all potential batch sizes. We also extend this to scenarios with constrained
active learning and constrained optimization, interpreting constraint
violations as reductions in the precision requirement, to subsequently adapt
batch construction. Through extensive experiments, we demonstrate that our
approach significantly enhances learning efficiency and flexibility in diverse
Bayesian batch active learning and Bayesian optimization applications.
