  Recent advances in molecular simulations allow the evaluation of previously
unattainable observables, such as rate constants for protein folding. However,
these calculations are usually computationally expensive and even significant
computing resources may result in a small number of independent estimates
spread over many orders of magnitude. Such small-sample, high "log-variance"
data are not readily amenable to analysis using the standard uncertainty (i.e.,
"standard error of the mean") because unphysical negative limits of confidence
intervals result. Bootstrapping, a natural alternative guaranteed to yield a
confidence interval within the minimum and maximum values, also exhibits a
striking systematic bias of the lower confidence limit in log space. As we
show, bootstrapping artifactually assigns high probability to improbably low
mean values. A second alternative, the Bayesian bootstrap strategy, does not
suffer from the same deficit and is more logically consistent with the type of
confidence interval desired. The Bayesian bootstrap provides uncertainty
intervals that are more reliable than those from the standard bootstrap method,
but must be used with caution nevertheless. Neither standard nor Bayesian
bootstrapping can overcome the intrinsic challenge of under-estimating the mean
from small-size, high log-variance samples. Our conclusions are based on
extensive analysis of model distributions and re-analysis of multiple
independent atomistic simulations. Although we only analyze rate constants,
similar considerations will apply to related calculations, potentially
including highly non-linear averages like the Jarzynski relation.
