  We review approaches to statistical inference based on randomization.
Permutation tests are treated as an important special case. Under a certain
group invariance property, referred to as the ``randomization hypothesis,''
randomization tests achieve exact control of the Type I error rate in finite
samples. Although this unequivocal precision is very appealing, the range of
problems that satisfy the randomization hypothesis is somewhat limited. We show
that randomization tests are often asymptotically, or approximately, valid and
efficient in settings that deviate from the conditions required for
finite-sample error control. When randomization tests fail to offer even
asymptotic Type 1 error control, their asymptotic validity may be restored by
constructing an asymptotically pivotal test statistic. Randomization tests can
then provide exact error control for tests of highly structured hypotheses with
good performance in a wider class of problems. We give a detailed overview of
several prominent applications of randomization tests, including two-sample
permutation tests, regression, and conformal inference.
