  Though black-box predictors are state-of-the-art for many complex tasks, they
often fail to properly quantify predictive uncertainty and may provide
inappropriate predictions for unfamiliar data. Instead, we can learn more
reliable models by letting them either output a prediction set or abstain when
the uncertainty is high. We propose training these selective prediction-set
models using an uncertainty-aware loss minimization framework, which unifies
ideas from decision theory and robust maximum likelihood. Moreover, since
black-box methods are not guaranteed to output well-calibrated prediction sets,
we show how to calculate point estimates and confidence intervals for the true
coverage of any selective prediction-set model, as well as a uniform mixture of
K set models obtained from K-fold sample-splitting. When applied to predicting
in-hospital mortality and length-of-stay for ICU patients, our model
outperforms existing approaches on both in-sample and out-of-sample age groups,
and our recalibration method provides accurate inference for prediction set
coverage.
