  In human-AI collaboration systems for critical applications, in order to
ensure minimal error, users should set an operating point based on model
confidence to determine when the decision should be delegated to human experts.
Samples for which model confidence is lower than the operating point would be
manually analysed by experts to avoid mistakes. Such systems can become truly
useful only if they consider two aspects: models should be confident only for
samples for which they are accurate, and the number of samples delegated to
experts should be minimized. The latter aspect is especially crucial for
applications where available expert time is limited and expensive, such as
healthcare. The trade-off between the model accuracy and the number of samples
delegated to experts can be represented by a curve that is similar to an ROC
curve, which we refer to as confidence operating characteristic (COC) curve. In
this paper, we argue that deep neural networks should be trained by taking into
account both accuracy and expert load and, to that end, propose a new
complementary loss function for classification that maximizes the area under
this COC curve. This promotes simultaneously the increase in network accuracy
and the reduction in number of samples delegated to humans. We perform
experiments on multiple computer vision and medical image datasets for
classification. Our results demonstrate that the proposed loss improves
classification accuracy and delegates less number of decisions to experts,
achieves better out-of-distribution samples detection and on par calibration
performance compared to existing loss functions.
