  Sampling methods such as Stratified Random Sampling can be used to select
representative samples of schools for randomized controlled trials of
educational interventions. However, these methods may still yield external
validity bias when participation by schools is voluntary and participation
decisions are associated with unobserved variables. This paper offers a new
sampling method called Stratified Random Sampling with Quotas. Under this
method, quotas are set to avoid including too many schools of a particular
type, as defined by auxiliary variables that are unobserved in the sampling
frame, but whose population distribution can be estimated from external data.
Our simulations find that when the auxiliary variables affect whether or not a
school participates in the study, quotas set based on those variables reduce
external validity bias. These results suggest that when auxiliary data are
available on strong impact moderators for the target population, these data can
be used to address non-ignorable self-selection by schools into randomized
controlled trials.
