  Data-driven, machine learning (ML) models of atomistic interactions are often
based on flexible and non-physical functions that can relate nuanced aspects of
atomic arrangements into predictions of energies and forces. As a result, these
potentials are as good as the training data (usually results of so-called ab
initio simulations) and we need to make sure that we have enough information
for a model to become sufficiently accurate, reliable and transferable. The
main challenge stems from the fact that descriptors of chemical environments
are often sparse high-dimensional objects without a well-defined continuous
metric. Therefore, it is rather unlikely that any ad hoc method of choosing
training examples will be indiscriminate, and it will be easy to fall into the
trap of confirmation bias, where the same narrow and biased sampling is used to
generate train- and test- sets. We will demonstrate that classical concepts of
statistical planning of experiments and optimal design can help to mitigate
such problems at a relatively low computational cost. The key feature of the
method we will investigate is that they allow us to assess the informativeness
of data (how much we can improve the model by adding/swapping a training
example) and verify if the training is feasible with the current set before
obtaining any reference energies and forces -- a so-called off-line approach.
In other words, we are focusing on an approach that is easy to implement and
doesn't require sophisticated frameworks that involve automated access to
high-performance computational (HPC).
