  Conformal prediction (CP) generates a set of predictions for a given test
sample such that the prediction set almost always contains the true label
(e.g., 99.5\% of the time). CP provides comprehensive predictions on possible
labels of a given test sample, and the size of the set indicates how certain
the predictions are (e.g., a set larger than one is `uncertain'). Such distinct
properties of CP enable effective collaborations between human experts and
medical AI models, allowing efficient intervention and quality check in
clinical decision-making. In this paper, we propose a new method called
Reliable-Region-Based Conformal Prediction (RR-CP), which aims to impose a
stronger statistical guarantee so that the user-specified error rate (e.g.,
0.5\%) can be achieved in the test time, and under this constraint, the size of
the prediction set is optimized (to be small). We consider a small prediction
set size an important measure only when the user-specified error rate is
achieved. Experiments on five public datasets show that our RR-CP performs
well: with a reasonably small-sized prediction set, it achieves the
user-specified error rate (e.g., 0.5\%) significantly more frequently than
exiting CP methods.
