  Inferring causal effects on long-term outcomes using short-term surrogates is
crucial to rapid innovation. However, even when treatments are randomized and
surrogates fully mediate their effect on outcomes, it's possible that we get
the direction of causal effects wrong due to confounding between surrogates and
outcomes -- a situation famously known as the surrogate paradox. The
availability of many historical experiments offer the opportunity to instrument
for the surrogate and bypass this confounding. However, even as the number of
experiments grows, two-stage least squares has non-vanishing bias if each
experiment has a bounded size, and this bias is exacerbated when most
experiments barely move metrics, as occurs in practice. We show how to
eliminate this bias using cross-fold procedures, JIVE being one example, and
construct valid confidence intervals for the long-term effect in new
experiments where long-term outcome has not yet been observed. Our methodology
further allows to proxy for effects not perfectly mediated by the surrogates,
allowing us to handle both confounding and effect leakage as violations of
standard statistical surrogacy conditions.
