  We construct validation designs $Z_m$ aimed at estimating the integrated
squared prediction error of a given design $X_n$. Our approach is based on the
minimization of a maximum mean discrepancy for a particular kernel, conditional
on $X_n$, so that sequences of nested validation designs can be constructed
incrementally by kernel herding. Numerical experiments show that key features
for a good validation design are its space-filling properties, in order to fill
the holes left by $X_n$ and properly explore the whole design space, and the
suitable weighting of its points, since evaluations far from $X_n$ tend to
overestimate the global error. A dedicated weighting method, based on a
particular kernel, is proposed. Numerical simulations with random functions
show the superiority the method over more traditional validation based on
random designs, low-discrepancy sequences, or leave-one-out cross validation.
