  Weighted least squares fitting to a database of quantum mechanical
calculations can determine the optimal parameters of empirical potential
models. While algorithms exist to provide optimal potential parameters for a
given fitting database of structures and their structure property functions,
and to estimate prediction errors using Bayesian sampling, defining an optimal
fitting database based on potential predictions remains elusive. A testing set
of structures and their structure property functions provides an empirical
measure of potential transferability. Here, we propose an objective function
for fitting databases based on testing set errors. The objective function
allows the optimization of the weights in a fitting database, the assessment of
the inclusion or removal of structures in the fitting database, or the
comparison of two different fitting databases. To showcase this technique, we
consider an example Lennard-Jones potential for Ti, where modeling multiple
complicated crystal structures is difficult for a radial pair potential. The
algorithm finds different optimal fitting databases, depending on the objective
function of potential prediction error for a testing set.
