  We demonstrate that distributed block coordinate descent can quickly solve
kernel regression and classification problems with millions of data points.
Armed with this capability, we conduct a thorough comparison between the full
kernel, the Nystr\"om method, and random features on three large classification
tasks from various domains. Our results suggest that the Nystr\"om method
generally achieves better statistical accuracy than random features, but can
require significantly more iterations of optimization. Lastly, we derive new
rates for block coordinate descent which support our experimental findings when
specialized to kernel methods.
