  Statistical divergences quantify the difference between probability
distributions, thereby allowing for multiple uses in machine-learning. However,
a fundamental challenge of these quantities is their estimation from empirical
samples since the underlying distributions of the data are usually unknown. In
this work, we propose a divergence inspired by the Jensen-Shannon divergence
which avoids the estimation of the probability density functions. Our approach
embeds the data in an reproducing kernel Hilbert space (RKHS) where we
associate data distributions with uncentered covariance operators in this
representation space. Therefore, we name this measure the representation
Jensen-Shannon divergence (RJSD). We provide an estimator from empirical
covariance matrices by explicitly mapping the data to an RKHS using Fourier
features. This estimator is flexible, scalable, differentiable, and suitable
for minibatch-based optimization problems. Additionally, we provide an
estimator based on kernel matrices without an explicit mapping to the RKHS. We
provide consistency convergence results for the proposed estimator. Moreover,
we demonstrate that this quantity is a lower bound on the Jensen-Shannon
divergence, leading to a variational approach to estimate it with theoretical
guarantees. We leverage the proposed divergence to train generative networks,
where our method mitigates mode collapse and encourages samples diversity.
Additionally, RJSD surpasses other state-of-the-art techniques in multiple
two-sample testing problems, demonstrating superior performance and reliability
in discriminating between distributions.
