  When we use simulation to evaluate the performance of a stochastic system,
the simulation often contains input distributions estimated from real-world
data; therefore, there is both simulation and input uncertainty in the
performance estimates. Ignoring either source of uncertainty underestimates the
overall statistical error. Simulation uncertainty can be reduced by additional
computation (e.g., more replications). Input uncertainty can be reduced by
collecting more real-world data, when feasible. This paper proposes an approach
to quantify overall statistical uncertainty when the simulation is driven by
independent parametric input distributions; specifically, we produce a
confidence interval that accounts for both simulation and input uncertainty by
using a metamodel-assisted bootstrapping approach. The input uncertainty is
measured via bootstrapping, an equation-based stochastic kriging metamodel
propagates the input uncertainty to the output mean, and both simulation and
metamodel uncertainty are derived using properties of the metamodel. A variance
decomposition is proposed to estimate the relative contribution of input to
overall uncertainty; this information indicates whether the overall uncertainty
can be significantly reduced through additional simulation alone. Asymptotic
analysis provides theoretical support for our approach, while an empirical
study demonstrates that it has good finite-sample performance.
