  The statistical literature is known to be inconsistent in the use of the
terms "permutation test" and "randomization test". Several authors succesfully
argue that these terms should be used to refer to two distinct classes of tests
and that there are major conceptual differences between these classes. The
present paper explains an important difference in mathematical reasoning
between these classes: a permutation test fundamentally requires that the set
of permutations has a group structure, in the algebraic sense; the reasoning
behind a randomization test is not based on such a group structure and it is
possible to use an experimental design that does not correspond to a group. In
particular, we can use a randomization scheme where the number of possible
treatment patterns is larger than in standard experimental designs. This leads
to exact \emph{p}-values of improved resolution, providing increased power for
very small significance levels, at the cost of decreased power for larger
significance levels. We discuss applications in randomized trials and
elsewhere. Further, we explain that Fisher's famous Lady Tasting Tea
experiment, which is commonly referred to as the first permutation test, is in
fact a randomization test. This distinction is important to avoid confusion and
invalid tests.
