  In many business applications, including online marketing and customer churn
prevention, randomized controlled trials (RCT's) are conducted to investigate
on the effect of specific treatment (coupon offers, advertisement
mailings,...). Such RCT's allow for the estimation of average treatment effects
as well as the training of (uplift) models for the heterogeneity of treatment
effects between individuals. The problem with these RCT's is that they are
costly and this cost increases with the number of individuals included into the
RCT. For this reason, there is research how to conduct experiments involving a
small number of individuals while still obtaining precise treatment effect
estimates. We contribute to this literature a heteroskedasticity-aware
stratified sampling (HS) scheme, which leverages the fact that different
individuals have different noise levels in their outcome and precise treatment
effect estimation requires more observations from the "high-noise" individuals
than from the "low-noise" individuals. By theory as well as by empirical
experiments, we demonstrate that our HS-sampling yields significantly more
precise estimates of the ATE, improves uplift models and makes their evaluation
more reliable compared to RCT data sampled completely randomly. Due to the
relative ease of application and the significant benefits, we expect
HS-sampling to be valuable in many real-world applications.
