  The role of numerical accuracy in training and evaluating neural
network-based potential energy surfaces is examined for different experimental
observables. For observables that require third- and fourth-order derivatives
of the total energy with respect to Cartesian coordinates single-precision
arithmetics as is typically used in ML-based approaches is insufficient and
leads to roughness of the underlying PES as is explicitly demonstrated.
Increasing the numerical accuracy to double-precision yields a smooth PES with
higher-order derivatives that are numerically stable and yield meaningful
anharmonic frequencies and tunneling splitting as is demonstrated for H$_2$CO
and malonaldehyde. For molecular dynamics simulations, which only require
first-order derivatives, single-precision arithmetics appears to be sufficient,
though.
