  We prove that Centered Kernel Alignment (CKA) based on a Gaussian RBF kernel
converges to linear CKA in the large-bandwidth limit. We show that convergence
onset is sensitive to the geometry of the feature representations, and that
representation eccentricity bounds the range of bandwidths for which Gaussian
CKA behaves nonlinearly.
