  Divergence measures play a central role and become increasingly essential in
deep learning, yet efficient measures for multiple (more than two)
distributions are rarely explored. This becomes particularly crucial in areas
where the simultaneous management of multiple distributions is both inevitable
and essential. Examples include clustering, multi-source domain adaptation or
generalization, and multi-view learning, among others. While computing the mean
of pairwise distances between any two distributions is a prevalent method to
quantify the total divergence among multiple distributions, it is imperative to
acknowledge that this approach is not straightforward and necessitates
significant computational resources. In this study, we introduce a new
divergence measure tailored for multiple distributions named the generalized
Cauchy-Schwarz divergence (GCSD). Additionally, we furnish a kernel-based
closed-form sample estimator, making it convenient and straightforward to use
in various machine-learning applications. Finally, we explore its profound
implications in the realm of deep learning by applying it to tackle two
thoughtfully chosen machine-learning tasks: deep clustering and multi-source
domain adaptation. Our extensive experimental investigations confirm the
robustness and effectiveness of GCSD in both scenarios. The findings also
underscore the innovative potential of GCSD and its capability to significantly
propel machine learning methodologies that necessitate the quantification of
multiple distributions.
