  Simulation metamodeling refers to the construction of lower-fidelity models
to represent input-output relations using few simulation runs. Stochastic
kriging, which is based on Gaussian process, is a versatile and common
technique for such a task. However, this approach relies on specific model
assumptions and could encounter scalability challenges. In this paper, we study
an alternative metamodeling approach using prediction intervals to capture the
uncertainty of simulation outputs. We cast the metamodeling task as an
empirical constrained optimization framework to train prediction intervals that
attain accurate prediction coverage and narrow width. We specifically use
neural network to represent these intervals and discuss procedures to
approximately solve this optimization problem. We also present an adaptation of
conformal prediction tools as another approach to construct prediction
intervals for metamodeling. Lastly, we present a validation machinery and show
how our method can enjoy a distribution-free finite-sample guarantee on the
prediction performance. We demonstrate and compare our proposed approaches with
existing methods including stochastic kriging through numerical examples.
