  Gaussian Process Regression is a popular nonparametric regression method
based on Bayesian principles that provides uncertainty estimates for its
predictions. However, these estimates are of a Bayesian nature, whereas for
some important applications, like learning-based control with safety
guarantees, frequentist uncertainty bounds are required. Although such rigorous
bounds are available for Gaussian Processes, they are too conservative to be
useful in applications. This often leads practitioners to replacing these
bounds by heuristics, thus breaking all theoretical guarantees. To address this
problem, we introduce new uncertainty bounds that are rigorous, yet practically
useful at the same time. In particular, the bounds can be explicitly evaluated
and are much less conservative than state of the art results. Furthermore, we
show that certain model misspecifications lead to only graceful degradation. We
demonstrate these advantages and the usefulness of our results for
learning-based control with numerical examples.
