  A benefit of randomized experiments is that covariate distributions of
treatment and control groups are balanced on average, resulting in simple
unbiased estimators for treatment effects. However, it is possible that a
particular randomization yields covariate imbalances that researchers want to
address in the analysis stage through adjustment or other methods. Here we
present a randomization test that conditions on covariate balance by only
considering treatment assignments that are similar to the observed one in terms
of covariate balance. Previous conditional randomization tests have only
allowed for categorical covariates, while our randomization test allows for any
type of covariate. Through extensive simulation studies, we find that our
conditional randomization test is more powerful than unconditional
randomization tests and other conditional tests. Furthermore, we find that our
conditional randomization test is valid (1) unconditionally across levels of
covariate balance, and (2) conditional on particular levels of covariate
balance. Meanwhile, unconditional randomization tests are valid for (1) but not
(2). Finally, we find that our conditional randomization test is similar to a
randomization test that uses a model-adjusted test statistic.
