  We establish the inferential properties of the mean-difference estimator for
the average treatment effect in randomized experiments where each unit in a
population is randomized to one of two treatments and then units within
treatment groups are randomly sampled. The properties of this estimator are
well-understood in the experimental design scenario where first units are
randomly sampled and then treatment is randomly assigned, but not for the
aforementioned scenario where the sampling and treatment assignment stages are
reversed. We find that the inferential properties of the mean-difference
estimator under this experimental design scenario are identical to those under
the more common sample-first-randomize-second design. This finding will bring
some clarifications about sampling-based randomized designs for causal
inference, particularly for settings where there is a finite super-population.
Finally, we explore to what extent pre-treatment measurements can be used to
improve upon the mean-difference estimator for this
randomize-first-sample-second design. Unfortunately, we find that pre-treatment
measurements are often unhelpful in improving the precision of average
treatment effect estimators under this design, unless a large number of
pre-treatment measurements that are highly associative with the post-treatment
measurements can be obtained. We confirm these results using a simulation study
based on a real experiment in nanomaterials.
