  This paper develops a finite-population, design-based theory of uncertainty
for studying quasi-experimental settings in the social sciences. In our
framework, treatment is determined by stochastic idiosyncratic factors, but
individuals may differ in their probability of receiving treatment in ways
unknown to the researcher, thus allowing for rich selection into treatment. We
derive formulas for the bias of common estimators (including
difference-in-means and difference-in-differences), and provide conditions
under which they are unbiased for an interpretable causal estimand (e.g.
analogs to the ATE or ATT). We further show that when the finite population is
large, conventional standard errors are valid but typically conservative for
the variance of the estimator over the randomization distribution. An
interesting feature of our framework is that conventional standard errors tend
to become more conservative when treatment probabilities vary more across
units, i.e. when there is more selection into treatment. This conservativeness
can (at least partially) mitigate undercoverage of conventional confidence
intervals when the estimator is biased because of selection. Our results also
have implications for the appropriate level to cluster standard errors, and for
the analysis of linear covariate adjustment and instrumental variables in
quasi-experimental settings.
