  Random Fourier features (RFF) represent one of the most popular and
wide-spread techniques in machine learning to scale up kernel algorithms.
Despite the numerous successful applications of RFFs, unfortunately, quite
little is understood theoretically on their optimality and limitations of their
performance. Only recently, precise statistical-computational trade-offs have
been established for RFFs in the approximation of kernel values, kernel ridge
regression, kernel PCA and SVM classification. Our goal is to spark the
investigation of optimality of RFF-based approximations in tasks involving not
only function values but derivatives, which naturally lead to optimization
problems with kernel derivatives. Particularly, in this paper, we focus on the
approximation quality of RFFs for kernel derivatives and prove that the
existing finite-sample guarantees can be improved exponentially in terms of the
domain where they hold, using recent tools from unbounded empirical process
theory. Our result implies that the same approximation guarantee is attainable
for kernel derivatives using RFF as achieved for kernel values.
