  In this paper, we consider the nonparametric least square regression in a
Reproducing Kernel Hilbert Space (RKHS). We propose a new randomized algorithm
that has optimal generalization error bounds with respect to the square loss,
closing a long-standing gap between upper and lower bounds. Moreover, we show
that our algorithm has faster finite-time and asymptotic rates on problems
where the Bayes risk with respect to the square loss is small. We state our
results using standard tools from the theory of least square regression in
RKHSs, namely, the decay of the eigenvalues of the associated integral operator
and the complexity of the optimal predictor measured through the integral
operator.
