  We derive uniformly most powerful (UMP) tests for simple and one-sided
hypotheses for a population proportion within the framework of Differential
Privacy (DP), optimizing finite sample performance. We show that in general, DP
hypothesis tests can be written in terms of linear constraints, and for
exchangeable data can always be expressed as a function of the empirical
distribution. Using this structure, we prove a 'Neyman-Pearson lemma' for
binomial data under DP, where the DP-UMP only depends on the sample sum. Our
tests can also be stated as a post-processing of a random variable, whose
distribution we coin ''Truncated-Uniform-Laplace'' (Tulap), a generalization of
the Staircase and discrete Laplace distributions. Furthermore, we obtain exact
$p$-values, which are easily computed in terms of the Tulap random variable.
  Using the above techniques, we show that our tests can be applied to give
uniformly most accurate one-sided confidence intervals and optimal confidence
distributions. We also derive uniformly most powerful unbiased (UMPU) two-sided
tests, which lead to uniformly most accurate unbiased (UMAU) two-sided
confidence intervals. We show that our results can be applied to
distribution-free hypothesis tests for continuous data. Our simulation results
demonstrate that all our tests have exact type I error, and are more powerful
than current techniques.
