  This paper examines experimental design procedures used to develop surrogates
of computational models, exploring the interplay between experimental designs
and approximation algorithms. We focus on two widely used approximation
approaches, Gaussian process (GP) regression and non-intrusive polynomial
approximation. First, we introduce algorithms for minimizing a posterior
integrated variance (IVAR) design criterion for GP regression. Our formulation
treats design as a continuous optimization problem that can be solved with
gradient-based methods on complex input domains, without resorting to greedy
approximations. We show that minimizing IVAR in this way yields point sets with
good interpolation properties, and that it enables more accurate GP regression
than designs based on entropy minimization or mutual information maximization.
Second, using a Mercer kernel/eigenfunction perspective on GP regression, we
identify conditions under which GP regression coincides with pseudospectral
polynomial approximation. Departures from these conditions can be understood as
changes either to the kernel or to the experimental design itself. We then show
how IVAR-optimal designs, while sacrificing discrete orthogonality of the
kernel eigenfunctions, can yield lower approximation error than orthogonalizing
point sets. Finally, we compare the performance of adaptive Gaussian process
regression and adaptive pseudospectral approximation for several classes of
target functions, identifying features that are favorable to the GP + IVAR
approach.
