  We prove rates of convergence in the statistical sense for kernel-based least
squares regression using a conjugate gradient algorithm, where regularization
against overfitting is obtained by early stopping. This method is directly
related to Kernel Partial Least Squares, a regression method that combines
supervised dimensionality reduction with least squares projection. The rates
depend on two key quantities: first, on the regularity of the target regression
function and second, on the intrinsic dimensionality of the data mapped into
the kernel space. Lower bounds on attainable rates depending on these two
quantities were established in earlier literature, and we obtain upper bounds
for the considered method that match these lower bounds (up to a log factor) if
the true regression function belongs to the reproducing kernel Hilbert space.
If this assumption is not fulfilled, we obtain similar convergence rates
provided additional unlabeled data are available. The order of the learning
rates match state-of-the-art results that were recently obtained for least
squares support vector machines and for linear regularization operators.
