  A statistical hypothesis test determines whether a hypothesis should be
rejected based on samples from populations. In particular, randomized
controlled experiments (or A/B testing) that compare population means using,
e.g., t-tests, have been widely deployed in technology companies to aid in
making data-driven decisions. Samples used in these tests are collected from
users and may contain sensitive information. Both the data collection and the
testing process may compromise individuals' privacy. In this paper, we study
how to conduct hypothesis tests to compare population means while preserving
privacy. We use the notation of local differential privacy (LDP), which has
recently emerged as the main tool to ensure each individual's privacy without
the need of a trusted data collector. We propose LDP tests that inject noise
into every user's data in the samples before collecting them (so users do not
need to trust the data collector), and draw conclusions with bounded type-I
(significance level) and type-II errors (1 - power). Our approaches can be
extended to the scenario where some users require LDP while some are willing to
provide exact data. We report experimental results on real-world datasets to
verify the effectiveness of our approaches.
