  Real-life data are often non-IID due to complex distributions and
interactions, and the sensitivity to the distribution of samples can differ
among learning models. Accordingly, a key question for any supervised or
unsupervised model is whether the probability distributions of two given
datasets can be considered identical. To address this question, we introduce
R-divergence, designed to assess model-oriented distribution discrepancies. The
core insight is that two distributions are likely identical if their optimal
hypothesis yields the same expected risk for each distribution. To estimate the
distribution discrepancy between two datasets, R-divergence learns a minimum
hypothesis on the mixed data and then gauges the empirical risk difference
between them. We evaluate the test power across various unsupervised and
supervised tasks and find that R-divergence achieves state-of-the-art
performance. To demonstrate the practicality of R-divergence, we employ
R-divergence to train robust neural networks on samples with noisy labels.
