  Selective classification enables models to make predictions only when they
are sufficiently confident, aiming to enhance safety and reliability, which is
important in high-stakes scenarios. Previous methods mainly use deep neural
networks and focus on modifying the architecture of classification layers to
enable the model to estimate the confidence of its prediction. This work
provides a generalization bound for selective classification, disclosing that
optimizing feature layers helps improve the performance of selective
classification. Inspired by this theory, we propose to explicitly improve the
selective classification model at the feature level for the first time, leading
to a novel Confidence-aware Contrastive Learning method for Selective
Classification, CCL-SC, which similarizes the features of homogeneous instances
and differentiates the features of heterogeneous instances, with the strength
controlled by the model's confidence. The experimental results on typical
datasets, i.e., CIFAR-10, CIFAR-100, CelebA, and ImageNet, show that CCL-SC
achieves significantly lower selective risk than state-of-the-art methods,
across almost all coverage degrees. Moreover, it can be combined with existing
methods to bring further improvement.
