  Regularization is an essential element of virtually all kernel methods for
nonparametric regression problems. A critical factor in the effectiveness of a
given kernel method is the type of regularization that is employed. This
article compares and contrasts members from a general class of regularization
techniques, which notably includes ridge regression and principal component
regression. We derive an explicit finite-sample risk bound for
regularization-based estimators that simultaneously accounts for (i) the
structure of the ambient function space, (ii) the regularity of the true
regression function, and (iii) the adaptability (or qualification) of the
regularization. A simple consequence of this upper bound is that the risk of
the regularization-based estimators matches the minimax rate in a variety of
settings. The general bound also illustrates how some regularization techniques
are more adaptable than others to favorable regularity properties that the true
regression function may possess. This, in particular, demonstrates a striking
difference between kernel ridge regression and kernel principal component
regression. Our theoretical results are supported by numerical experiments.
