<p>The answer appears to be in your quotes:</p>
<p>&quot;The intuition here is that only units that are alike on unobservables and observables would follow a similar trajectory pre-treatment.&quot;</p>
<p>That intuition isn't necessarily true.  There could <strong>always</strong> be unobserved confounders which hamper the analysis. Nothing renders you immune from that.</p>
<p>However, once you are done controlling for all observable factors, it seems unclear what else can be done.  Most good econometric analyses contain a large narrative component indicating to readers reasons why a given DiD (or other treatment method) is plausibly randomized or the nonrandom component is sufficiently accounted for.  This element is not statistical in nature - it is a persuasive narrative designed to convince (or illustrate to the reader) that all unobserved factors are controlled for, something that cannot be done by pointing to observed data.</p>
<p>If all observable factors are controlled for, and some respectable attempt at identifying uncontrolled factors has been shown, then I would expect any remaining naysayers should at least be able to name specifically what unobserved factor is causing the problem. Otherwise, a critic could always suppose <em>some</em> omitted variable exists, and then add no further commentary.</p>

CC BY-SA 4.0
2022-01-06T23:50:24.967Z
via [Wikipedia](https://en.wikipedia.org/wiki/Difference_in_differences#Assumptions): "The Achilles' heel of DID is when something other than the treatment changes in one group but not the other at the same time as the treatment, implying a violation of the parallel trend assumption." While the stated benefit of synthetic control is that it does not require the parallel trends assumption, this admission seems to imply that synthetic control is no better than DID in situations where there might be "treatment changes in one group but not the other at the same time as the treatment."
Daycent
CC BY-SA 4.0
2022-01-06T23:51:53.017Z
your answer makes sense, though it is not entirely satisfactory -- it seems the onus should be on the researcher to ensure that there are no unobserved factors that affect the results.
Daycent
CC BY-SA 4.0
2022-01-07T01:02:00.373Z
That's unfortunately not possible. The researcher can address many unobserved factors, but not all conceivable ones. One cannot disprove the existence of a thing, particularly an unobserved thing.
RegressForward
CC BY-SA 4.0
2022-01-06T23:04:00.173Z
2022-01-07T01:12:56.227Z
2022-01-07T01:12:56.227Z
RegressForward
<p>From Scott Cunningham in <em>Mixtape</em>, on propensity score matching:</p>
<blockquote>
<p>propensity score matching has not seen as wide adoption among
economists as in other non-experimental methods like regression
discontinuity or difference-in-differences. The most common reason
given for this is that economists are oftentimes skeptical that the conditional independence assumption
can be achieved in any data set—almost as an article of faith. This is
because for many applications, economists as a group are usually more
concerned about selection on unobservables than they are selection on
observables, and as such, they reach for matching methods less often.</p>
</blockquote>
<p>Synthetic control seems simply to be a more sophisticated method of matching, where instead of matching each outcome with one observation per treated observation, we are matching based on a weighted average of observations.</p>
<p>Again, from Cunningham (on synthetic control):</p>
<blockquote>
<p>What about unobserved factors? Comparative case studies are
complicated by unmeasured factors affecting the outcome of interest as
well as heterogeneity in the effect of observed and unobserved
factors. Abadie, Diamond, and Hainmueller (2010) note that if the
number of pre-intervention periods in the data is “large,” then
matching on pre-intervention outcomes can allow us to control for the
heterogeneous responses to multiple unobserved factors. The intuition
here is that <em>only units that are alike on unobservables and
observables would follow a similar trajectory pre-treatment.</em></p>
</blockquote>
<p><strong>However, if I am weighting my observations by training on the pre-treatment period, is it not possible that there is some variable <span class="math-container">$X$</span> that will still confound and bias my estimates?</strong></p>
<p><em><strong>e.g.</strong></em> Say I want to evaluate the effectiveness of a medical intervention on lowering hospital costs. The likelihood of receiving treatment is based on a doctor's subjective assessment of how sick the patient <em>will be</em> in the future (unobserved confounding variable <span class="math-container">$X$</span>). Now, I so happen to have 3 covariates on my patients: sex, age, and income. I use that data to select my optimal weights for the post-treatment period.
So, in this case:</p>
<ol>
<li><p>Isn't synthetic control only applicable in this case insofar as my
covariates are able to roughly approximate <span class="math-container">$X$</span>? In this case, is it
not entirely possible that they will <em>not</em>, even if I achieve a
tight cross-validated fit in the pre-treatment period? -- i.e. the doctor knows something about the patients' future condition that my 3 variables cannot describe, <em>even if</em> my 3 variables are able to accurately reflect patients' costs pre-treatment.</p>
</li>
<li><p>Additionally, how do I know that the relationship between the
outcome of interest and my covariates do not change across time? Let's say sex, age, and income so happen to not be relevant for
predicting patient hospital costs in 2022, even though they were
quite effective in 2019 (potentially because of a confounding
variable like a global pandemic). Wouldn't synthetic control
completely misattribute the causal effect of the intervention? Or would this be moot using randomization inference?</p>
</li>
</ol>

CC BY-SA 4.0
2022-01-06T01:33:16.770Z
I don't believe the above quote (or any other quote) says you are immune to unobserved confounders under synthetic controls. Could you narrow it down?
RegressForward
<econometrics><applied-econometrics><synthetic-control>
Why does synthetic control not suffer from backdoor confounders?
Daycent
# Why does synthetic control not suffer from backdoor confounders?

(Asked by: username_0 on 2022-01-05)

From Scott Cunningham in *Mixtape*, on propensity score matching:

> propensity score matching has not seen as wide adoption among
> economists as in other non-experimental methods like regression
> discontinuity or difference-in-differences. The most common reason
> given for this is that economists are oftentimes skeptical that the conditional independence assumption
> can be achieved in any data set—almost as an article of faith. This is
> because for many applications, economists as a group are usually more
> concerned about selection on unobservables than they are selection on
> observables, and as such, they reach for matching methods less often.

Synthetic control seems simply to be a more sophisticated method of matching, where instead of matching each outcome with one observation per treated observation, we are matching based on a weighted average of observations.

Again, from Cunningham (on synthetic control):

> What about unobserved factors? Comparative case studies are
> complicated by unmeasured factors affecting the outcome of interest as
> well as heterogeneity in the effect of observed and unobserved
> factors. Abadie, Diamond, and Hainmueller (2010) note that if the
> number of pre-intervention periods in the data is “large,” then
> matching on pre-intervention outcomes can allow us to control for the
> heterogeneous responses to multiple unobserved factors. The intuition
> here is that *only units that are alike on unobservables and
> observables would follow a similar trajectory pre-treatment.*

**However, if I am weighting my observations by training on the pre-treatment period, is it not possible that there is some variable $X$ that will still confound and bias my estimates?**

***e.g.*** Say I want to evaluate the effectiveness of a medical intervention on lowering hospital costs. The likelihood of receiving treatment is based on a doctor's subjective assessment of how sick the patient *will be* in the future (unobserved confounding variable $X$). Now, I so happen to have 3 covariates on my patients: sex, age, and income. I use that data to select my optimal weights for the post-treatment period.
So, in this case:

1. Isn't synthetic control only applicable in this case insofar as my
   covariates are able to roughly approximate $X$? In this case, is it
   not entirely possible that they will *not*, even if I achieve a
   tight cross-validated fit in the pre-treatment period? -- i.e. the doctor knows something about the patients' future condition that my 3 variables cannot describe, *even if* my 3 variables are able to accurately reflect patients' costs pre-treatment.
2. Additionally, how do I know that the relationship between the
   outcome of interest and my covariates do not change across time? Let's say sex, age, and income so happen to not be relevant for
   predicting patient hospital costs in 2022, even though they were
   quite effective in 2019 (potentially because of a confounding
   variable like a global pandemic). Wouldn't synthetic control
   completely misattribute the causal effect of the intervention? Or would this be moot using randomization inference?

**Tags:** econometrics, applied-econometrics, synthetic-control

> **Comments:**
> **username_1**: I don't believe the above quote (or any other quote) says you are immune to unobserved confounders under synthetic controls. Could you narrow it down?  

---

## 1 answers

### Answer by username_1 on 2022-01-06. Score: 3 | Accepted answer

The answer appears to be in your quotes:

"The intuition here is that only units that are alike on unobservables and observables would follow a similar trajectory pre-treatment."

That intuition isn't necessarily true. There could **always** be unobserved confounders which hamper the analysis. Nothing renders you immune from that.

However, once you are done controlling for all observable factors, it seems unclear what else can be done. Most good econometric analyses contain a large narrative component indicating to readers reasons why a given DiD (or other treatment method) is plausibly randomized or the nonrandom component is sufficiently accounted for. This element is not statistical in nature - it is a persuasive narrative designed to convince (or illustrate to the reader) that all unobserved factors are controlled for, something that cannot be done by pointing to observed data.

If all observable factors are controlled for, and some respectable attempt at identifying uncontrolled factors has been shown, then I would expect any remaining naysayers should at least be able to name specifically what unobserved factor is causing the problem. Otherwise, a critic could always suppose *some* omitted variable exists, and then add no further commentary.

> **Comments:**
> **username_0**: via [Wikipedia](https://en.wikipedia.org/wiki/Difference_in_differences#Assumptions): "The Achilles' heel of DID is when something other than the treatment changes in one group but not the other at the same time as the treatment, implying a violation of the parallel trend assumption." While the stated benefit of synthetic control is that it does not require the parallel trends assumption, this admission seems to imply that synthetic control is no better than DID in situations where there might be "treatment changes in one group but not the other at the same time as the treatment."  
> **username_0**: your answer makes sense, though it is not entirely satisfactory -- it seems the onus should be on the researcher to ensure that there are no unobserved factors that affect the results.  
> **username_1**: That's unfortunately not possible. The researcher can address many unobserved factors, but not all conceivable ones. One cannot disprove the existence of a thing, particularly an unobserved thing.  

---

