  When the data are sparse, optimization of hyperparameters of the kernel in
Gaussian process regression by the commonly used maximum likelihood estimation
(MLE) criterion often leads to overfitting. We show that choosing
hyperparameters (in this case, kernel length parameter and regularization
parameter) based on a criterion of the completeness of the basis in the
corresponding linear regression problem is superior to MLE. We show that this
is facilitated by the use of high-dimensional model representation (HDMR)
whereby a low-order HDMR representation can provide reliable reference
functions and large synthetic test data sets needed for basis parameter
optimization even when the original data are few.
