  Instrumental variable (IV) strategies are widely used in political science to
establish causal relationships. However, the identifying assumptions required
by an IV design are demanding, and it remains challenging for researchers to
assess their validity. In this paper, we replicate 67 papers published in three
top journals in political science during 2010-2022 and identify several
troubling patterns. First, researchers often overestimate the strength of their
IVs due to non-i.i.d. errors, such as a clustering structure. Second, the most
commonly used t-test for the two-stage-least-squares (2SLS) estimates often
severely underestimates uncertainty. Using more robust inferential methods, we
find that around 19-30% of the 2SLS estimates in our sample are underpowered.
Third, in the majority of the replicated studies, the 2SLS estimates are much
larger than the ordinary-least-squares estimates, and their ratio is negatively
correlated with the strength of the IVs in studies where the IVs are not
experimentally generated, suggesting potential violations of unconfoundedness
or the exclusion restriction. To help researchers avoid these pitfalls, we
provide a checklist for better practice.
