  The aim of the present work is to construct prediction intervals via the
Jackknife and the Jackknife+ whose coverage probability conditional on the
training data is close to its nominal level in finite samples and can be
asymptotically valid in high-dimensions. The main innovation is twofold:
Firstly, we generalize the results of Steinberger and Leeb (2023) for the
Jackknife to a non-continuous response distribution and to the case of
non-linear models. Secondly, we show that under a stability assumption the
prediction intervals via a Jackknife+-approach are closely related to their
Jackknife analogues and therefore share a similar conditional coverage
probability. More specifically, this work is split into four parts: in the
first part we introduce the $\varepsilon$-variational divergence in order to
link the prediction interval's coverage probability to the accuracy of
estimating the distribution of the prediction error. In the second part, we
define an approach based on the Jackknife for the estimation of the prediction
error's distribution conditional on the training data. We show that the
prediction error's distribution can be estimated consistently if two conditions
are fulfilled: the prediction error should be bounded in probability and the
prediction algorithm should satisfy a stability condition. In particular, we
show that under these two conditions the prediction intervals based on the
Jackknife are asymptotically valid. In the fourth part we draw a connection
between the Jackknife and the Jackknife+ and show that corresponding prediction
intervals are closely related to each other given the stability condition. In
the last part we show that under mild assumptions the conditions for
asymptotically valid prediction intervals based on the Jackknife or the
Jackknife+ are fulfilled for several predictors.
