<p>[a synthesis of Evangelos, Michael and my comments / posts]</p>
<p>You are right if you have a simple, two-time period diff-in-diff and you believe parallel trends is true OR if you have staggered/heterogenous timing with comparator group only being the never treated.</p>
<p>But, if we sometimes compare the treated with different groups of pre-treated (as in staggered/heterogenous timing with comparator group including the pre-treated), then we could get a negative coefficient with anticipation effects.</p>
<p>As a motivating example, say we're evaluating effects of cash transfers, and we only have two time periods. Some (very poor individuals) anticipate they will receive a cash transfer, and therefore spend before receiving the cash transfer to help improve their livelihoods. Others do not anticipate and spend to a lesser degree after receiving the cash transfer. Then, since treatment timing is homogeneous (rich and poor treated at the same time), the diff in diff is:</p>
<p>[Average of treated post] - [Average of treated pre] - ([Average of control post] - [Average of control pre])</p>
<p>The treated comprise both rich and poor. Anticipatory behavior generally means that [Average of treated pre] is larger.</p>
<p>However, you'd get a negative effect if somehow there was heterogeneity in treatment timing. Suppose instead that the rich were treated first, while the poor were untreated. The poor anticipated treatement, and increased spending while untreated. Then the comparison between treated and pre-treated will result in the pre-treated having increased spending a lot (due to anticipation) and a potentially negative estimate. This gets resolved if you restrict the comparator group to the never-treated.</p>
<p>I'd still be wary, however. It is possible that anticipatory behavior means that [Average of treated pre] is smaller (take moral hazard with health insurance, for example).</p>
<p>Moreover, treatment effects can incorporate (1) a change in levels and (2) a change in trends. If treatment (and anticipation of treatment), causes a change in trends, then parallel trends will fail, causing problems. In general, pre-trends due to anticipatory behavior can't be distinguished from pre-trends due to any other reason, which would imply a failure of parallel trends.</p>
<p>Also, nice review article of the literature arxiv.org/abs/2201.01194.</p>

CC BY-SA 4.0
2022-05-07T15:33:23.867Z
Also, useful https://mixtape.scunning.com/difference-in-differences.html
Evangelos Con
CC BY-SA 4.0
2022-05-06T22:49:16.290Z
2022-05-06T22:49:16.290Z
Daycent
<p>If some units anticipate the policy change and act before others, then it seems to me that this problem is almost isomorphic to staggered adoption DiD design. And that literature suggests that DiD with heterogeneous effects can get the estimates very wrong (e.g. wrong sign). So, the type of anticipation you are describing would be a serious concern---for example, those anticipating are probably the ones that have the most to gain.</p>
<p>So, I doubt that it will nicely offset publication bias.</p>

CC BY-SA 4.0
2022-05-06T07:47:41.790Z
The paper being referenced is this one, https://www.sciencedirect.com/science/article/pii/S030440762030378X
Michael Gmeiner
CC BY-SA 4.0
2022-05-06T07:48:09.863Z
If there is heterogeneous treatment timing and heterogeneous treatment effects, then the two-way fixed effects estimator has bias.
Michael Gmeiner
CC BY-SA 4.0
2022-05-06T07:49:32.933Z
If there isn't staggered adoption, but there is anticipatory behavior, it is not obvious to me that this could result in the wrong sign. I need to think about this...
Michael Gmeiner
CC BY-SA 4.0
2022-05-06T12:16:26.063Z
The original question was about _some_ (and not all) treated unites anticipate the treatment and respond. That's what makes me think it's similar to staggered treatment. In the simplest case, I think would agree with your other answer. But heterogeneous effects are (rightly) most dominating the literature.
Evangelos Con
CC BY-SA 4.0
2022-05-06T12:17:36.543Z
Also, nice review article of the literature https://arxiv.org/abs/2201.01194.
Evangelos Con
CC BY-SA 4.0
2022-05-06T13:27:42.620Z
I think this is the answer. IF the units that anticipated the treatment had greater treatment effects than those units that did NOT anticipate the treatment (e.g. those that stood to benefit the most), then that might bias the DiD coefficient so much that comparing units that didn't anticipate to those that did anticipate would result in a negative coefficient.
Daycent
CC BY-SA 4.0
2022-05-06T13:30:19.410Z
for example, say we're evaluating effects of cash transfers. some (very poor individuals) anticipate they will receive a cash transfer, and therefore spend *before* receiving the cash transfer to help improve their livelihoods. Others do not anticipate and spend to a lesser degree *after* receiving the cash transfer. Using DiD could result in negative treatment effects if there are sufficient numbers of people in the former group.
Daycent
CC BY-SA 4.0
2022-05-06T13:46:10.680Z
Perhaps one way to alleviate this is if you have a covariate that you think predicts whether an individual will anticipate (e.g. income), you could use that covariate to shift certain people into an earlier time/separate group.
Daycent
CC BY-SA 4.0
2022-05-06T13:59:29.983Z
I thought a core tenant of the Sun and Abraham contamination was that the binary variables for being a certain time period after treatment differ in absolute time across treated individuals (due to heterogeneous treatment timing). 
Is that the case here? I thought there was homogeneous treatment timing, just with anticipation among some treated units. That seems fundamentally different and I'm not sure you can just apply the Sun and Abraham result.
Michael Gmeiner
CC BY-SA 4.0
2022-05-06T14:04:18.967Z
In Daycent's example, is there a control group without cash transfers? Are rich and poor treated at the same time? If so, the diff in diff is: [Average of treated post] - [Average of treated pre] -  ([Average of control post] - [Average of control pre]).

The treated comprise both rich and poor. 

Anticipatory behavior means that [Average of treated pre]  is larger. However you'd only get a negative effect if somehow there was heterogeneity in treatment timing.
Michael Gmeiner
CC BY-SA 4.0
2022-05-06T14:25:32.850Z
That is, suppose the rich were treated first, while the poor were untreated. The poor anticipated treatement, and increased spending while untreated. Then the comparison between treated and untreated will result in the untreated having increased spending a lot (due to anticipation) and a potentially negative estimate.

But if you have homogeneous treatment timing, I'm not convinced this problem persists.
Michael Gmeiner
CC BY-SA 4.0
2022-05-06T14:42:50.127Z
Hi Michael, thanks again for the response. I think I now understand. What you're saying is that if we *always* compare the treated with the *never*-treated, then anticipation effects will always decrease the treatment effect towards a null result (2 time periods, or in staggered timing with comparator group only being the never treated).
Daycent
CC BY-SA 4.0
2022-05-06T14:42:55.377Z
But, if we sometimes compare the treated with different groups of pre-treated (as in staggered timing with comparator group including the pre-treated), then we could get a negative coefficient with anticipation effects. If you write up that answer, I'll accept that as correct.
Daycent
CC BY-SA 4.0
2022-05-06T14:56:12.540Z
@MichaelGmeiner You last comment is pretty much how I understand Daycent's original Q.

The way the problem is set up, anticipation works as if some people get the treatment before others. Thus, staggered timing and not homogeneous. So, I don't think  plain DiD works very well here.

However, I obviously agree with the comments about homogeneous timing and the newer estimators. I think the conversation drifted a bit, but I'll close with not fully agreeing with the point about publication bias even with homogeneous timing or newer estimators,
Evangelos Con
CC BY-SA 4.0
2022-05-05T22:38:58.783Z
2022-05-05T22:38:58.783Z
Evangelos Con
<p>If you have a simple, two-time period diff-in-diff and you believe parallel trends is true, I think you are right.</p>
<p>I'd still be wary, however. Treatment effects can incorporate (1) a change in levels and (2) a change in trends. If treatment (and anticipation of treatment), causes a change in trends, then parallel trends will fail, causing problems.</p>

CC BY-SA 4.0
2022-05-06T13:57:40.263Z
I'm not convinced Evangelos's answer is correct. In your specification, all the units have the same treatment time correct? There's no heterogeneity in treatment time, just some have anticipation?

The paper that Evangelos cited has the core issue of heterogeneity in treatment time. If you don't have that, then I'm not sure that answer is correct.
Michael Gmeiner
CC BY-SA 4.0
2022-05-06T14:44:29.177Z
I originally was considering the general case (both heterogeneity and homogeneity in treatment timing). my apologies for not clarifying in the question.
Daycent
CC BY-SA 4.0
2022-05-06T07:46:43.757Z
2022-05-17T04:13:31.633Z
2022-05-17T04:13:31.633Z
Michael Gmeiner
<p>I am trying to implement a difference in difference model.
Suppose that some treated units anticipate the treatment.
Then, that would mean that they respond to the treatment <em>prior</em> to the treatment actually occuring.
Since the treatment effect would then be present in the &quot;pre&quot; period, it seems like this would almost always bias the relative post-period treatment effect downwards in magnitude.</p>
<p>In this sense, it never really matters if there is an anticipation effect. If there <em>is</em> an anticipation effect, all it does is decrease my Type 1 error rate and increase my Type 2 error rate. Given publication bias often results in a high Type 1 error rate (I discover some treatment effect that doesn't exist), isn't the &quot;anticipation effect&quot; really a moot point?</p>
<p>Thanks in advance!</p>

CC BY-SA 4.0
2022-05-06T15:02:24.237Z
There's no reason to assume the two biases are of remotely equal sizes, or even estimable sizes in the population at large. Nor will there (usually) be evidence that in any particular case one bias isn't dominating.   When considering opposing biases, it's prudent to assume you're wrong in both directions.
RegressForward
<econometrics><difference-in-difference>
Do anticipation effects almost always bias the DiD ATE or ATT towards a null result and increase standard errors?
Daycent
# Do anticipation effects almost always bias the DiD ATE or ATT towards a null result and increase standard errors?

(Asked by: username_0 on 2022-05-05)

I am trying to implement a difference in difference model.
Suppose that some treated units anticipate the treatment.
Then, that would mean that they respond to the treatment *prior* to the treatment actually occuring.
Since the treatment effect would then be present in the "pre" period, it seems like this would almost always bias the relative post-period treatment effect downwards in magnitude.

In this sense, it never really matters if there is an anticipation effect. If there *is* an anticipation effect, all it does is decrease my Type 1 error rate and increase my Type 2 error rate. Given publication bias often results in a high Type 1 error rate (I discover some treatment effect that doesn't exist), isn't the "anticipation effect" really a moot point?

Thanks in advance!

**Tags:** econometrics, difference-in-difference

> **Comments:**
> **username_1**: There's no reason to assume the two biases are of remotely equal sizes, or even estimable sizes in the population at large. Nor will there (usually) be evidence that in any particular case one bias isn't dominating.   When considering opposing biases, it's prudent to assume you're wrong in both directions.  

---

## 3 answers

### Answer by username_0 on 2022-05-06. Score: 1 | Accepted answer

[a synthesis of Evangelos, Michael and my comments / posts]

You are right if you have a simple, two-time period diff-in-diff and you believe parallel trends is true OR if you have staggered/heterogenous timing with comparator group only being the never treated.

But, if we sometimes compare the treated with different groups of pre-treated (as in staggered/heterogenous timing with comparator group including the pre-treated), then we could get a negative coefficient with anticipation effects.

As a motivating example, say we're evaluating effects of cash transfers, and we only have two time periods. Some (very poor individuals) anticipate they will receive a cash transfer, and therefore spend before receiving the cash transfer to help improve their livelihoods. Others do not anticipate and spend to a lesser degree after receiving the cash transfer. Then, since treatment timing is homogeneous (rich and poor treated at the same time), the diff in diff is:

[Average of treated post] - [Average of treated pre] - ([Average of control post] - [Average of control pre])

The treated comprise both rich and poor. Anticipatory behavior generally means that [Average of treated pre] is larger.

However, you'd get a negative effect if somehow there was heterogeneity in treatment timing. Suppose instead that the rich were treated first, while the poor were untreated. The poor anticipated treatement, and increased spending while untreated. Then the comparison between treated and pre-treated will result in the pre-treated having increased spending a lot (due to anticipation) and a potentially negative estimate. This gets resolved if you restrict the comparator group to the never-treated.

I'd still be wary, however. It is possible that anticipatory behavior means that [Average of treated pre] is smaller (take moral hazard with health insurance, for example).

Moreover, treatment effects can incorporate (1) a change in levels and (2) a change in trends. If treatment (and anticipation of treatment), causes a change in trends, then parallel trends will fail, causing problems. In general, pre-trends due to anticipatory behavior can't be distinguished from pre-trends due to any other reason, which would imply a failure of parallel trends.

Also, nice review article of the literature arxiv.org/abs/2201.01194.

> **Comments:**
> **username_2**: Also, useful https://mixtape.scunning.com/difference-in-differences.html  

---

### Answer by username_2 on 2022-05-05. Score: -1

If some units anticipate the policy change and act before others, then it seems to me that this problem is almost isomorphic to staggered adoption DiD design. And that literature suggests that DiD with heterogeneous effects can get the estimates very wrong (e.g. wrong sign). So, the type of anticipation you are describing would be a serious concern---for example, those anticipating are probably the ones that have the most to gain.

So, I doubt that it will nicely offset publication bias.

> **Comments:**
> **username_3**: The paper being referenced is this one, https://www.sciencedirect.com/science/article/pii/S030440762030378X  
> **username_3**: If there is heterogeneous treatment timing and heterogeneous treatment effects, then the two-way fixed effects estimator has bias.  
> **username_3**: If there isn't staggered adoption, but there is anticipatory behavior, it is not obvious to me that this could result in the wrong sign. I need to think about this...  
> **username_2**: The original question was about _some_ (and not all) treated unites anticipate the treatment and respond. That's what makes me think it's similar to staggered treatment. In the simplest case, I think would agree with your other answer. But heterogeneous effects are (rightly) most dominating the literature.  
> **username_2**: Also, nice review article of the literature https://arxiv.org/abs/2201.01194.  
> **username_0**: I think this is the answer. IF the units that anticipated the treatment had greater treatment effects than those units that did NOT anticipate the treatment (e.g. those that stood to benefit the most), then that might bias the DiD coefficient so much that comparing units that didn't anticipate to those that did anticipate would result in a negative coefficient.  
> **username_0**: for example, say we're evaluating effects of cash transfers. some (very poor individuals) anticipate they will receive a cash transfer, and therefore spend *before* receiving the cash transfer to help improve their livelihoods. Others do not anticipate and spend to a lesser degree *after* receiving the cash transfer. Using DiD could result in negative treatment effects if there are sufficient numbers of people in the former group.  
> **username_0**: Perhaps one way to alleviate this is if you have a covariate that you think predicts whether an individual will anticipate (e.g. income), you could use that covariate to shift certain people into an earlier time/separate group.  
> **username_3**: I thought a core tenant of the Sun and Abraham contamination was that the binary variables for being a certain time period after treatment differ in absolute time across treated individuals (due to heterogeneous treatment timing).  Is that the case here? I thought there was homogeneous treatment timing, just with anticipation among some treated units. That seems fundamentally different and I'm not sure you can just apply the Sun and Abraham result.  
> **username_3**: In username_0's example, is there a control group without cash transfers? Are rich and poor treated at the same time? If so, the diff in diff is: [Average of treated post] - [Average of treated pre] -  ([Average of control post] - [Average of control pre]).  The treated comprise both rich and poor.   Anticipatory behavior means that [Average of treated pre]  is larger. However you'd only get a negative effect if somehow there was heterogeneity in treatment timing.  
> **username_3**: That is, suppose the rich were treated first, while the poor were untreated. The poor anticipated treatement, and increased spending while untreated. Then the comparison between treated and untreated will result in the untreated having increased spending a lot (due to anticipation) and a potentially negative estimate.  But if you have homogeneous treatment timing, I'm not convinced this problem persists.  
> **username_0**: Hi Michael, thanks again for the response. I think I now understand. What you're saying is that if we *always* compare the treated with the *never*-treated, then anticipation effects will always decrease the treatment effect towards a null result (2 time periods, or in staggered timing with comparator group only being the never treated).  
> **username_0**: But, if we sometimes compare the treated with different groups of pre-treated (as in staggered timing with comparator group including the pre-treated), then we could get a negative coefficient with anticipation effects. If you write up that answer, I'll accept that as correct.  
> **username_2**: @MichaelGmeiner You last comment is pretty much how I understand username_0's original Q.  The way the problem is set up, anticipation works as if some people get the treatment before others. Thus, staggered timing and not homogeneous. So, I don't think  plain DiD works very well here.  However, I obviously agree with the comments about homogeneous timing and the newer estimators. I think the conversation drifted a bit, but I'll close with not fully agreeing with the point about publication bias even with homogeneous timing or newer estimators,  

---

### Answer by username_3 on 2022-05-06. Score: -1

If you have a simple, two-time period diff-in-diff and you believe parallel trends is true, I think you are right.

I'd still be wary, however. Treatment effects can incorporate (1) a change in levels and (2) a change in trends. If treatment (and anticipation of treatment), causes a change in trends, then parallel trends will fail, causing problems.

> **Comments:**
> **username_3**: I'm not convinced Evangelos's answer is correct. In your specification, all the units have the same treatment time correct? There's no heterogeneity in treatment time, just some have anticipation?  The paper that Evangelos cited has the core issue of heterogeneity in treatment time. If you don't have that, then I'm not sure that answer is correct.  
> **username_0**: I originally was considering the general case (both heterogeneity and homogeneity in treatment timing). my apologies for not clarifying in the question.  

---

