  When observed decisions depend only on observed features, off-policy policy
evaluation (OPE) methods for sequential decision making problems can estimate
the performance of evaluation policies before deploying them. This assumption
is frequently violated due to unobserved confounders, unrecorded variables that
impact both the decisions and their outcomes. We assess robustness of OPE
methods under unobserved confounding by developing worst-case bounds on the
performance of an evaluation policy. When unobserved confounders can affect
every decision in an episode, we demonstrate that even small amounts of
per-decision confounding can heavily bias OPE methods. Fortunately, in a number
of important settings found in healthcare, policy-making, operations, and
technology, unobserved confounders may primarily affect only one of the many
decisions made. Under this less pessimistic model of one-decision confounding,
we propose an efficient loss-minimization-based procedure for computing
worst-case bounds, and prove its statistical consistency. On two simulated
healthcare examples---management of sepsis patients and developmental
interventions for autistic children---where this is a reasonable model of
confounding, we demonstrate that our method invalidates non-robust results and
provides meaningful certificates of robustness, allowing reliable selection of
policies even under unobserved confounding.
