  The presence of inequity is a fundamental problem in the outcomes of
decision-making systems, especially when human lives are at stake. Yet,
estimating notions of unfairness or inequity is difficult, particularly if they
rely on hard-to-measure concepts such as risk. Such measurements of risk can be
accurately obtained when no unobserved confounders have jointly influenced past
decisions and outcomes. However, in the real world, this assumption rarely
holds. In this paper, we show a surprising result that one can still give
meaningful bounds on treatment rates to high-risk individuals, even when
entirely eliminating or relaxing the assumption that all relevant risk factors
are observed. We use the fact that in many real-world settings (e.g., the
release of a new treatment) we have data from prior to any allocation to derive
unbiased estimates of risk. This result is of immediate practical interest: we
can audit unfair outcomes of existing decision-making systems in a principled
manner. For instance, in a real-world study of Paxlovid allocation, our
framework provably identifies that observed racial inequity cannot be explained
by unobserved confounders of the same strength as important observed
covariates.
