  This paper proposes a statistically optimal approach for learning a function
value using a confidence interval in a wide range of models, including general
non-parametric estimation of an expected loss described as a stochastic
programming problem or various SDE models. More precisely, we develop a
systematic construction of highly accurate confidence intervals by using a
moderate deviation principle-based approach. It is shown that the proposed
confidence intervals are statistically optimal in the sense that they satisfy
criteria regarding exponential accuracy, minimality, consistency,
mischaracterization probability, and eventual uniformly most accurate (UMA)
property. The confidence intervals suggested by this approach are expressed as
solutions to robust optimization problems, where the uncertainty is expressed
via the underlying moderate deviation rate function induced by the
data-generating process. We demonstrate that for many models these optimization
problems admit tractable reformulations as finite convex programs even when
they are infinite-dimensional.
