  The Difference in Difference (DiD) estimator is a popular estimator built on
the "parallel trends" assumption, which is an assertion that the treatment
group, absent treatment, would change "similarly" to the control group over
time. To bolster such a claim, one might generate a comparison group, via
matching, that is similar to the treated group with respect to pre-treatment
outcomes and/or pre-treatment covariates. Unfortunately, as has been previously
pointed out, this intuitively appealing approach also has a cost in terms of
bias. To assess the trade-offs of matching in our application, we first
characterize the bias of matching prior to a DiD analysis under a linear
structural model that allows for time-invariant observed and unobserved
confounders with time-varying effects on the outcome. Given our framework, we
verify that matching on baseline covariates generally reduces bias. We further
show how additionally matching on pre-treatment outcomes has both cost and
benefit. First, matching on pre-treatment outcomes partially balances
unobserved confounders, which mitigates some bias. This reduction is
proportional to the outcome's reliability, a measure of how coupled the
outcomes are with the latent covariates. Offsetting these gains, matching also
injects bias into the final estimate by undermining the second difference in
the DiD via a regression-to-the-mean effect. Consequently, we provide heuristic
guidelines for determining to what degree the bias reduction of matching is
likely to outweigh the bias cost. We illustrate our guidelines by reanalyzing a
principal turnover study that used matching prior to a DiD analysis and find
that matching on both the pre-treatment outcomes and observed covariates makes
the estimated treatment effect more credible.
