  Recently, there as been an increasing interest in the use of heavily
restricted randomization designs which enforces balance on observed covariates
in randomized controlled trials. However, when restrictions are strict, there
is a risk that the treatment effect estimator will have a very high mean
squared error. In this paper, we formalize this risk and propose a novel
combinatoric-based approach to describe and address this issue. First, we
validate our new approach by re-proving some known properties of complete
randomization and restricted randomization. Second, we propose a novel
diagnostic measure for restricted designs that only use the information
embedded in the combinatorics of the design. Third, we show that the variance
of the mean squared error of the difference-in-means estimator in a randomized
experiment is a linear function of this diagnostic measure. Finally, we
identify situations in which restricted designs can lead to an increased risk
of getting a high mean squared error and discuss how our diagnostic measure can
be used to detect such designs. Our results have implications for any
restricted randomization design and can be used to evaluate the trade-off
between enforcing balance on observed covariates and avoiding too restrictive
designs.
