  In recent years the development of machine learning (ML) potentials (MLP) has
become a very active field of research. Numerous approaches have been proposed,
which allow to perform extended simulations of large systems at a small
fraction of the computational costs of electronic structure calculations. The
key to the success of modern ML potentials is the close-to first principles
quality description of the atomic interactions. This accuracy is reached by
using very flexible functional forms in combination with high-level reference
data from electronic structure calculations. These data sets can include up to
hundreds of thousands of structures covering millions of atomic environments to
ensure that all relevant features of the potential energy surface are well
represented. The handling of such large data sets is nowadays becoming one of
the main challenges in the construction of ML potentials. In this paper we
present a method, the bin-and-hash (BAH) algorithm, to overcome this problem by
enabling the efficient identification and comparison of large numbers of
multidimensional vectors. Such vectors emerge in multiple contexts in the
construction of ML potentials. Examples are the comparison of local atomic
environments to identify and avoid unnecessary redundant information in the
reference data sets that is costly in terms of both the electronic structure
calculations as well as the training process, the assessment of the quality of
the descriptors used as structural fingerprints in many types of ML potentials,
and the detection of possibly unreliable data points. The BAH algorithm is
illustrated for the example of high-dimensional neural network potentials using
atom-centered symmetry functions for the geometrical description of the atomic
environments, but the method is general and can be combined with any current
type of ML potential.
