  We consider the task of evaluating policies of algorithmic resource
allocation through randomized controlled trials (RCTs). Such policies are
tasked with optimizing the utilization of limited intervention resources, with
the goal of maximizing the benefits derived. Evaluation of such allocation
policies through RCTs proves difficult, notwithstanding the scale of the trial,
because the individuals' outcomes are inextricably interlinked through resource
constraints controlling the policy decisions. Our key contribution is to
present a new estimator leveraging our proposed novel concept, that involves
retrospective reshuffling of participants across experimental arms at the end
of an RCT. We identify conditions under which such reassignments are
permissible and can be leveraged to construct counterfactual trials, whose
outcomes can be accurately ascertained, for free. We prove theoretically that
such an estimator is more accurate than common estimators based on sample means
-- we show that it returns an unbiased estimate and simultaneously reduces
variance. We demonstrate the value of our approach through empirical
experiments on synthetic, semi-synthetic as well as real case study data and
show improved estimation accuracy across the board.
