  Under the potential outcomes framework, causal effects are defined as
comparisons between potential outcomes under treatment and control. To infer
causal effects from randomized experiments, Neyman proposed to test the null
hypothesis of zero average causal effect (Neyman's null), and Fisher proposed
to test the null hypothesis of zero individual causal effect (Fisher's null).
Although the subtle difference between Neyman's null and Fisher's null has
caused lots of controversies and confusions for both theoretical and practical
statisticians, a careful comparison between the two approaches has been lacking
in the literature for more than eighty years. We fill in this historical gap by
making a theoretical comparison between them and highlighting an intriguing
paradox that has not been recognized by previous researchers. Logically,
Fisher's null implies Neyman's null. It is therefore surprising that, in actual
completely randomized experiments, rejection of Neyman's null does not imply
rejection of Fisher's null for many realistic situations, including the case
with constant causal effect. Furthermore, we show that this paradox also exists
in other commonly-used experiments, such as stratified experiments,
matched-pair experiments, and factorial experiments. Asymptotic analyses,
numerical examples, and real data examples all support this surprising
phenomenon. Besides its historical and theoretical importance, this paradox
also leads to useful practical implications for modern researchers.
