  Hypothesis testing is a central problem in statistical analysis, and there is
currently a lack of differentially private tests which are both statistically
valid and powerful. In this paper, we develop several new differentially
private (DP) nonparametric hypothesis tests. Our tests are based on
Kolmogorov-Smirnov, Kuiper, Cram\'er-von Mises, and Wasserstein test
statistics, which can all be expressed as a pseudo-metric on empirical
cumulative distribution functions (ecdfs), and can be used to test hypotheses
on goodness-of-fit, two samples, and paired data. We show that these test
statistics have low sensitivity, requiring minimal noise to satisfy DP. In
particular, we show that the sensitivity of these test statistics can be
expressed in terms of the base sensitivity, which is the pseudo-metric distance
between the ecdfs of adjacent databases and is easily calculated. The sampling
distribution of our test statistics are distribution-free under the null
hypothesis, enabling easy computation of $p$-values by Monte Carlo methods. We
show that in several settings, especially with small privacy budgets or
heavy-tailed data, our new DP tests outperform alternative nonparametric DP
tests.
