  Online experiments are a fundamental component of the development of
web-facing products. Given their large user-bases, even small product
improvements can have a large impact on user engagement or profits on an
absolute scale. As a result, accurately estimating the relative impact of these
changes is extremely important. I propose an approach based on an objective
Bayesian model to improve the sensitivity of percent change estimation in A/B
experiments. Leveraging pre-period information, this approach produces more
robust and accurate point estimates and up to 50% tighter credible intervals
than traditional methods. The R package abpackage provides an implementation of
the approach.
