  Prediction algorithms assign numbers to individuals that are popularly
understood as individual "probabilities" -- what is the probability of 5-year
survival after cancer diagnosis? -- and which increasingly form the basis for
life-altering decisions. Drawing on an understanding of computational
indistinguishability developed in complexity theory and cryptography, we
introduce Outcome Indistinguishability. Predictors that are Outcome
Indistinguishable yield a generative model for outcomes that cannot be
efficiently refuted on the basis of the real-life observations produced by
Nature. We investigate a hierarchy of Outcome Indistinguishability definitions,
whose stringency increases with the degree to which distinguishers may access
the predictor in question. Our findings reveal that Outcome
Indistinguishability behaves qualitatively differently than previously studied
notions of indistinguishability. First, we provide constructions at all levels
of the hierarchy. Then, leveraging recently-developed machinery for proving
average-case fine-grained hardness, we obtain lower bounds on the complexity of
the more stringent forms of Outcome Indistinguishability. This hardness result
provides the first scientific grounds for the political argument that, when
inspecting algorithmic risk prediction instruments, auditors should be granted
oracle access to the algorithm, not simply historical predictions.
