  Parameter settings profoundly impact the performance of machine learning
algorithms and laboratory experiments. The classical grid search or trial-error
methods are exponentially expensive in large parameter spaces, and Bayesian
optimization (BO) offers an elegant alternative for global optimization of
black box functions. In situations where the black box function can be
evaluated at multiple points simultaneously, batch Bayesian optimization is
used. Current batch BO approaches are restrictive in that they fix the number
of evaluations per batch, and this can be wasteful when the number of specified
evaluations is larger than the number of real maxima in the underlying
acquisition function. We present the Budgeted Batch Bayesian Optimization (B3O)
for hyper-parameter tuning and experimental design - we identify the
appropriate batch size for each iteration in an elegant way. To set the batch
size flexible, we use the infinite Gaussian mixture model (IGMM) for
automatically identifying the number of peaks in the underlying acquisition
functions. We solve the intractability of estimating the IGMM directly from the
acquisition function by formulating the batch generalized slice sampling to
efficiently draw samples from the acquisition function. We perform extensive
experiments for both synthetic functions and two real world applications -
machine learning hyper-parameter tuning and experimental design for alloy
hardening. We show empirically that the proposed B3O outperforms the existing
fixed batch BO approaches in finding the optimum whilst requiring a fewer
number of evaluations, thus saving cost and time.
