  Machine learning interatomic potentials (MLIPs) are promising surrogates for
quantum mechanics evaluations in ab-initio molecular dynamics simulations due
to their ability to reproduce the energy and force landscape within chemical
accuracy at four orders of magnitude less cost. While developing uncertainty
quantification (UQ) tools for MLIPs is critical to build production MLIP
datasets using active learning, only limited progress has been made and the
most robust method, ensembling, still shows low correlation between high error
and high uncertainty predictions. Here we develop a rigorous method rooted in
statistics for determining an error cutoff that distinguishes regions of high
and low UQ performance. The statistical cutoff illuminates that a main cause of
the poor UQ performance is due to the machine learning model already describing
the entire dataset and not having any datapoints with error greater than the
statistical error distribution. Second, we extend the statistical analysis to
create an interpretable connection between the error and uncertainty
distributions to predict an uncertainty cutoff separating high and low errors.
We showcase the statistical cutoff in active learning benchmarks on two
datasets of varying chemical complexity for three common UQ methods:
ensembling, sparse Gaussian processes, and latent distance metrics and compare
them to the true error and random sampling, showing that the statistical cutoff
is generalizable to a variety of different UQ methods and protocols and
performs similarly to using the true error. Importantly, we conclude that
utilizing this uncertainty cutoff enables using significantly lower cost
uncertainty quantification tools such as sparse gaussian processes and latent
distances compared to ensembling approaches for generating MLIP datasets at a
fraction of the cost.
