  Decision makers often need to rely on imperfect probabilistic forecasts.
While average performance metrics are typically available, it is difficult to
assess the quality of individual forecasts and the corresponding utilities. To
convey confidence about individual predictions to decision-makers, we propose a
compensation mechanism ensuring that the forecasted utility matches the
actually accrued utility. While a naive scheme to compensate decision-makers
for prediction errors can be exploited and might not be sustainable in the long
run, we propose a mechanism based on fair bets and online learning that
provably cannot be exploited. We demonstrate an application showing how
passengers could confidently optimize individual travel plans based on flight
delay probabilities estimated by an airline.
