  In this paper, we study the problem of estimator selection and
hyper-parameter tuning in off-policy evaluation. Although cross-validation is
the most popular method for model selection in supervised learning, off-policy
evaluation relies mostly on theory-based approaches, which provide only limited
guidance to practitioners. We show how to use cross-validation for off-policy
evaluation. This challenges a popular belief that cross-validation in
off-policy evaluation is not feasible. We evaluate our method empirically and
show that it addresses a variety of use cases.
