<p>I was wondering if there's a proven/known relationship between RI p-values and OLS p-values in RCT settings, where treatment is completely random. <a href="https://personal.lse.ac.uk/YoungA/ChannellingFisherQJE.pdf" rel="nofollow noreferrer">Alwyn Young's paper</a><sup>1</sup> shows many examples of OLS p-values that were smaller than RI p-values, but is this a coincidence or broadly more true?</p>
<p>And if RI p-values can be smaller than OLS p-values, what would be the reason for that? Young argues that high leverage points cause OLS standard errors/p-values to become smaller than they should be, so is there a similar explanation for why OLS errors/p-values would be too large compared to RI?</p>
<p>Thank you!</p>
<p><sup>1</sup> <sub>Young, Alwyn. &quot;Channeling fisher: Randomization tests and the statistical insignificance of seemingly significant experimental results.&quot; <em>The quarterly journal of economics</em> 134, no. 2 (2019): 557-598.</sub></p>

CC BY-SA 4.0
2024-02-06T22:46:56.173Z
It would be helpful if you provided a link or a more informative citation of the paper you are referring to.
Kitsune Cavalry
CC BY-SA 4.0
2024-02-07T15:09:49.463Z
Adding that now, thank you for flagging!
msc
CC BY-SA 4.0
2024-02-07T15:16:09.543Z
Even better to have both, so that one may learn what it is without having to download a pdf.
Richard Hardy
<econometrics><experimental-economics><randomized-experiment>
In RCT settings, are randomization inference p-values always greater than OLS p-values?
msc
# In RCT settings, are randomization inference p-values always greater than OLS p-values?

(Asked by: username_0 on 2024-02-06)

I was wondering if there's a proven/known relationship between RI p-values and OLS p-values in RCT settings, where treatment is completely random. [Alwyn Young's paper](https://personal.lse.ac.uk/YoungA/ChannellingFisherQJE.pdf)1 shows many examples of OLS p-values that were smaller than RI p-values, but is this a coincidence or broadly more true?

And if RI p-values can be smaller than OLS p-values, what would be the reason for that? Young argues that high leverage points cause OLS standard errors/p-values to become smaller than they should be, so is there a similar explanation for why OLS errors/p-values would be too large compared to RI?

Thank you!

1 Young, Alwyn. "Channeling fisher: Randomization tests and the statistical insignificance of seemingly significant experimental results." *The quarterly journal of economics* 134, no. 2 (2019): 557-598.

**Tags:** econometrics, experimental-economics, randomized-experiment

> **Comments:**
> **username_1**: It would be helpful if you provided a link or a more informative citation of the paper you are referring to.  
> **username_0**: Adding that now, thank you for flagging!  
> **username_2**: Even better to have both, so that one may learn what it is without having to download a pdf.  

---

## 0 answers

