  We address the calibration of a computationally expensive nuclear physics
model for which derivative information with respect to the fit parameters is
not readily available. Of particular interest is the performance of
optimization-based training algorithms when dozens, rather than millions or
more, of training data are available and when the expense of the model places
limitations on the number of concurrent model evaluations that can be
performed.
  As a case study, we consider the Fayans energy density functional model,
which has characteristics similar to many model fitting and calibration
problems in nuclear physics. We analyze hyperparameter tuning considerations
and variability associated with stochastic optimization algorithms and
illustrate considerations for tuning in different computational settings.
