  Unquantified sources of uncertainty in observational causal analyses can
break the integrity of the results. One would never want another analyst to
repeat a calculation with the same dataset, using a seemingly identical
procedure, only to find a different conclusion. However, as we show in this
work, there is a typical source of uncertainty that is essentially never
considered in observational causal studies: the choice of match assignment for
matched groups, that is, which unit is matched to which other unit before a
hypothesis test is conducted. The choice of match assignment is anything but
innocuous, and can have a surprisingly large influence on the causal
conclusions. Given that a vast number of causal inference studies test
hypotheses on treatment effects after treatment cases are matched with similar
control cases, we should find a way to quantify how much this extra source of
uncertainty impacts results. What we would really like to be able to report is
that \emph{no matter} which match assignment is made, as long as the match is
sufficiently good, then the hypothesis test result still holds. In this paper,
we provide methodology based on discrete optimization to create robust tests
that explicitly account for this possibility. We formulate robust tests for
binary and continuous data based on common test statistics as integer linear
programs solvable with common methodologies. We study the finite-sample
behavior of our test statistic in the discrete-data case. We apply our methods
to simulated and real-world datasets and show that they can produce useful
results in practical applied settings.
