  The Nystr\"om method is one of the most popular techniques for improving the
scalability of kernel methods. However, it has not yet been derived for kernel
PCA in line with classical PCA. In this paper we derive kernel PCA with the
Nystr\"om method, thereby providing one of the few available options to make
kernel PCA scalable. We further study its statistical accuracy through a
finite-sample confidence bound on the empirical reconstruction error compared
to the full method. The behaviours of the method and bound are illustrated
through computer experiments on multiple real-world datasets. As an application
of the method we present kernel principal component regression with the
Nystr\"om method, as an alternative to Nystr\"om kernel ridge regression for
efficient regularized regression with kernels.
