  We consider the problem of model multiplicity in downstream decision-making,
a setting where two predictive models of equivalent accuracy cannot agree on
the best-response action for a downstream loss function. We show that even when
the two predictive models approximately agree on their individual predictions
almost everywhere, it is still possible for their induced best-response actions
to differ on a substantial portion of the population. We address this issue by
proposing a framework that calibrates the predictive models with regard to both
the downstream decision-making problem and the individual probability
prediction. Specifically, leveraging tools from multi-calibration, we provide
an algorithm that, at each time-step, first reconciles the differences in
individual probability prediction, then calibrates the updated models such that
they are indistinguishable from the true probability distribution to the
decision-maker. We extend our results to the setting where one does not have
direct access to the true probability distribution and instead relies on a set
of i.i.d data to be the empirical distribution. Finally, we provide a set of
experiments to empirically evaluate our methods: compared to existing work, our
proposed algorithm creates a pair of predictive models with both improved
downstream decision-making losses and agrees on their best-response actions
almost everywhere.
