<p>There is no universal rule,  in principle you could get away even with the joint test showing significant differences at 10% but not at 5%. It all depends on specifics of your research. For example, if you have large sample, coefficients will be estimated with much higher precision so using 10% level would not be reasonable (although when it comes to testing parallel trend assumption it is rare to have more than 5-10 years of pre treatment data to check for the trend).</p>
<p>In addition, there is no bullet proof way of testing parallel trends, you can do not just one test but battery of tests to be more confident (see literature review on various ways of testing for parallel trends in <a href="https://oconnell.fas.harvard.edu/files/jroth/files/roth_pretrends_testing.pdf" rel="nofollow noreferrer">Roth 2019a</a> or <a href="https://scholar.harvard.edu/files/jroth/files/honestparalleltrends_main.pdf" rel="nofollow noreferrer">Rambachan &amp; Roth 2020</a>).</p>
<p>But generally there are no widely accepted benchmarks, some people still get  away with showcasing plots that show the variables sort of move together before intervention and without rigorous testing. It varies by subfield, and is context depended, my recommendation is to either look at what other people are doing in your subfield and apply similar tests, or potentially a bit more than that if you want to go extra mile.</p>

CC BY-SA 4.0
2021-09-10T07:08:18.230Z
Thanks a heap, @1muflon1, can I ask what does " **if you have large sample, coefficients will be estimated with much higher precision so using 10% level would not be reasonable** " mean? I understand large sample can lead to more precisive, but why it leads to "using 10% level would not be reasonable"? Because in parallel trend test, using 10% level is more rigid than using 5% level. Do you mean "with large sample, we should not test parallel at 5% level but we should use 10% benchmark generally" ?
Phil Nguyen
CC BY-SA 4.0
2021-09-10T07:11:49.523Z
" **in principle you could get away even with the joint test showing significant differences at 10% but not at 5%** " Do you mean 10% level (p-value >0.1) is quite a safe way to prove parallel trend assumption being satisfied right?
Phil Nguyen
CC BY-SA 4.0
2021-09-10T10:53:51.553Z
@Louise because the more precision you have the likelier it is that even small difference between two trends that is economically not very meaningful will happen to be statistically significantly different from zero unless you go for more strict level like 5 or 1%. Also yes I meant p values, you check p value against your desired significance level which in social science is usually either 10, 5 or 1% or in decimal form 0.1, 0,05 and 0.01
1muflon1
CC BY-SA 4.0
2021-09-10T21:09:29.770Z
so, you mean, with a large dataset (saying around 350,000 observations), there is no substantial difference between 10% or 5%, or, in other words, we can say the parallel trend is satisfied if the p-value of the joint null test even smaller than 0.1 but higher than 0.05 ?
Phil Nguyen
CC BY-SA 4.0
2021-09-10T21:12:51.663Z
@Louise no, I am saying that in a large dataset with 350k observations if the difference between trend is only significant at 10% or 0.1 then it might not be significant at all. With that many data points if some test is not significant at at least 5% or heck even 1% it is not really significant. Significance value tells you what is probability of committing type I error. With 350k observations your estimates should be estimated with enough precision that you should simply not tolerate chance of 10% type I error. But in smaller dataset you might be willing to do that since there is a lot of
1muflon1
CC BY-SA 4.0
2021-09-10T21:13:55.853Z
noise there so something statistically significant at 10% might actually truly indicate actual difference
1muflon1
CC BY-SA 4.0
2021-09-10T22:32:01.813Z
"With that many data points if some test is not significant at at least 5% or heck even 1% it is not really significant", I know it is quite strange but can I ask the reference for this one then? In this case we should draw the picture to prove our point? (In parallel trend test case)
Phil Nguyen
CC BY-SA 4.0
2021-09-10T22:43:04.580Z
@Louise what exactly you want reference for? The fact that precision of the estimates increases with sample size is well known and you can find it any textbook, for example the formula for t-test is given by $t= \beta/(s/\sqrt{n})$ which is clearly increasing in number of observations n, so for example if beta is 0.5 s=1 with 4 observations the t stat would be 1 which would not be significant at any level, but with the same numbers just with higher number of observations let’s say n=25, the t statistics would 2.5 which would be significant at 5% level. Yet nothing changed here we have the same
1muflon1
CC BY-SA 4.0
2021-09-10T22:45:30.587Z
Point estimate, the same variability of that estimate just more data. Hence you need to take this fact in account no matter what empirical research you do, generally for smaller sample sizes using 10% level might be ok but for larger ones you should use 5% or even 1% definitely not 10%
1muflon1
CC BY-SA 4.0
2021-09-10T22:50:18.573Z
1muflon1, an awesome answer, thanks a heap, have a great day!
Phil Nguyen
CC BY-SA 4.0
2021-09-10T06:40:04.693Z
2021-09-10T06:40:04.693Z
1muflon1
<p>There's no widely accepted benchmark. The idea is to present as much evidence as you can to convince your audience that the parallel trend is satisfied. The least you can do (and most of the time is done in economics papers) is to plot the trend and see if there's any discernible divergent trend before the intervention. It also depends on your unit of analysis; if your unit of analysis is smaller than the unit of treatment (for instance, if you have individual data, but your treatment is happening at the city level), you can present a balance table showing that your treatment and control groups are similar on observable characteristics. The joint test you posted can also be helpful.</p>

CC BY-SA 4.0
2021-09-10T13:38:52.963Z
2021-09-10T13:38:52.963Z
Econlover
<p>For parallel trend assumption in Difference-in-Differences (DiD), normally which benchmark we normally use to judge whether the parallel assumption is being satisfied?</p>
<p>From this answer from <a href="https://economics.stackexchange.com/questions/47157/what-are-the-hypothesis-and-results-explaination-of-joint-null-test">@1muflon1</a>, it seems that the p-value of the joint null test of coefficients before the event date higher than 0,1 is the benchmark to say the parallel test is satisfied (there is no difference between treatment and control group before the event date).</p>
<p>I am wondering it is right-thinking? And I am wondering if there is any reference for this benchmark.</p>

<econometrics><reference-request>
What are the benchmarks for parallel trend test in Difference-in-Differences?
Phil Nguyen
# What are the benchmarks for parallel trend test in Difference-in-Differences?

(Asked by: username_0 on 2021-09-09)

For parallel trend assumption in Difference-in-Differences (DiD), normally which benchmark we normally use to judge whether the parallel assumption is being satisfied?

From this answer from [@username_1](https://economics.stackexchange.com/questions/47157/what-are-the-hypothesis-and-results-explaination-of-joint-null-test), it seems that the p-value of the joint null test of coefficients before the event date higher than 0,1 is the benchmark to say the parallel test is satisfied (there is no difference between treatment and control group before the event date).

I am wondering it is right-thinking? And I am wondering if there is any reference for this benchmark.

**Tags:** econometrics, reference-request


---

## 2 answers

### Answer by username_1 on 2021-09-10. Score: 1 | Accepted answer

There is no universal rule, in principle you could get away even with the joint test showing significant differences at 10% but not at 5%. It all depends on specifics of your research. For example, if you have large sample, coefficients will be estimated with much higher precision so using 10% level would not be reasonable (although when it comes to testing parallel trend assumption it is rare to have more than 5-10 years of pre treatment data to check for the trend).

In addition, there is no bullet proof way of testing parallel trends, you can do not just one test but battery of tests to be more confident (see literature review on various ways of testing for parallel trends in [Roth 2019a](https://oconnell.fas.harvard.edu/files/jroth/files/roth_pretrends_testing.pdf) or [Rambachan & Roth 2020](https://scholar.harvard.edu/files/jroth/files/honestparalleltrends_main.pdf)).

But generally there are no widely accepted benchmarks, some people still get away with showcasing plots that show the variables sort of move together before intervention and without rigorous testing. It varies by subfield, and is context depended, my recommendation is to either look at what other people are doing in your subfield and apply similar tests, or potentially a bit more than that if you want to go extra mile.

> **Comments:**
> **username_0**: Thanks a heap, @username_1, can I ask what does " **if you have large sample, coefficients will be estimated with much higher precision so using 10% level would not be reasonable** " mean? I understand large sample can lead to more precisive, but why it leads to "using 10% level would not be reasonable"? Because in parallel trend test, using 10% level is more rigid than using 5% level. Do you mean "with large sample, we should not test parallel at 5% level but we should use 10% benchmark generally" ?  
> **username_0**: " **in principle you could get away even with the joint test showing significant differences at 10% but not at 5%** " Do you mean 10% level (p-value >0.1) is quite a safe way to prove parallel trend assumption being satisfied right?  
> **username_1**: @Louise because the more precision you have the likelier it is that even small difference between two trends that is economically not very meaningful will happen to be statistically significantly different from zero unless you go for more strict level like 5 or 1%. Also yes I meant p values, you check p value against your desired significance level which in social science is usually either 10, 5 or 1% or in decimal form 0.1, 0,05 and 0.01  
> **username_0**: so, you mean, with a large dataset (saying around 350,000 observations), there is no substantial difference between 10% or 5%, or, in other words, we can say the parallel trend is satisfied if the p-value of the joint null test even smaller than 0.1 but higher than 0.05 ?  
> **username_1**: @Louise no, I am saying that in a large dataset with 350k observations if the difference between trend is only significant at 10% or 0.1 then it might not be significant at all. With that many data points if some test is not significant at at least 5% or heck even 1% it is not really significant. Significance value tells you what is probability of committing type I error. With 350k observations your estimates should be estimated with enough precision that you should simply not tolerate chance of 10% type I error. But in smaller dataset you might be willing to do that since there is a lot of  
> **username_1**: noise there so something statistically significant at 10% might actually truly indicate actual difference  
> **username_0**: "With that many data points if some test is not significant at at least 5% or heck even 1% it is not really significant", I know it is quite strange but can I ask the reference for this one then? In this case we should draw the picture to prove our point? (In parallel trend test case)  
> **username_1**: @Louise what exactly you want reference for? The fact that precision of the estimates increases with sample size is well known and you can find it any textbook, for example the formula for t-test is given by $t= \beta/(s/\sqrt{n})$ which is clearly increasing in number of observations n, so for example if beta is 0.5 s=1 with 4 observations the t stat would be 1 which would not be significant at any level, but with the same numbers just with higher number of observations let’s say n=25, the t statistics would 2.5 which would be significant at 5% level. Yet nothing changed here we have the same  
> **username_1**: Point estimate, the same variability of that estimate just more data. Hence you need to take this fact in account no matter what empirical research you do, generally for smaller sample sizes using 10% level might be ok but for larger ones you should use 5% or even 1% definitely not 10%  
> **username_0**: username_1, an awesome answer, thanks a heap, have a great day!  

---

### Answer by username_2 on 2021-09-10. Score: 1

There's no widely accepted benchmark. The idea is to present as much evidence as you can to convince your audience that the parallel trend is satisfied. The least you can do (and most of the time is done in economics papers) is to plot the trend and see if there's any discernible divergent trend before the intervention. It also depends on your unit of analysis; if your unit of analysis is smaller than the unit of treatment (for instance, if you have individual data, but your treatment is happening at the city level), you can present a balance table showing that your treatment and control groups are similar on observable characteristics. The joint test you posted can also be helpful.


---

