  Atomistic molecular simulations are a powerful way to make quantitative
predictions, but the accuracy of these predictions depends entirely on the
quality of the forcefield employed. While experimental measurements of
fundamental physical properties offer a straightforward approach for evaluating
forcefield quality, the bulk of this information has been tied up in formats
that are not machine-readable. Compiling benchmark datasets of physical
properties from non-machine-readable sources require substantial human effort
and is prone to accumulation of human errors, hindering the development of
reproducible benchmarks of forcefield accuracy. Here, we examine the
feasibility of benchmarking atomistic forcefields against the NIST ThermoML
data archive of physicochemical measurements, which aggregates thousands of
experimental measurements in a portable, machine-readable, self-annotating
format. As a proof of concept, we present a detailed benchmark of the
generalized Amber small molecule forcefield (GAFF) using the AM1-BCC charge
model against measurements (specifically bulk liquid densities and static
dielectric constants at ambient pressure) automatically extracted from the
archive, and discuss the extent of available data. The results of this
benchmark highlight a general problem with fixed-charge forcefields in the
representation low dielectric environments such as those seen in binding
cavities or biological membranes.
