  From the social sciences to machine learning, it has been well documented
that metrics to be optimized are not always aligned with social welfare. In
healthcare, Dranove et al. (2003) showed that publishing surgery mortality
metrics actually harmed the welfare of sicker patients by increasing provider
selection behavior. We analyze the incentive misalignments that arise from such
average treated outcome metrics, and show that the incentives driving treatment
decisions would align with maximizing total patient welfare if the metrics (i)
accounted for counterfactual untreated outcomes and (ii) considered total
welfare instead of averaging over treated patients. Operationalizing this, we
show how counterfactual metrics can be modified to behave reasonably in
patient-facing ranking systems. Extending to realistic settings when providers
observe more about patients than the regulatory agencies do, we bound the decay
in performance by the degree of information asymmetry between principal and
agent. In doing so, our model connects principal-agent information asymmetry
with unobserved heterogeneity in causal inference.
