  In machine learning applications, the reliability of predictions is
significant for assisted decision and risk control. As an effective framework
to quantify the prediction reliability, conformal prediction (CP) was developed
with the CPKNN (CP with kNN). However, the conventional CPKNN suffers from high
variance and bias and long computational time as the feature dimensionality
increases. To address these limitations, a new CP framework-conformal
prediction with shrunken centroids (CPSC) is proposed. It regularizes the class
centroids to attenuate the irrelevant features and shrink the sample space for
predictions and reliability quantification. To compare CPKNN and CPSC, we
employed them in the classification of 12 categories of alternative herbal
medicine with electronic nose as a case and assessed them in two tasks: 1)
offline prediction: the training set was fixed and the accuracy on the testing
set was evaluated; 2) online prediction with data augmentation: they filtered
unlabeled data to augment the training data based on the prediction reliability
and the final accuracy of testing set was compared. The result shows that CPSC
significantly outperformed CPKNN in both two tasks: 1) CPSC reached a
significantly higher accuracy with lower computation cost, and with the same
credibility output, CPSC generally achieves a higher accuracy; 2) the data
augmentation process with CPSC robustly manifested a statistically significant
improvement in prediction accuracy with different reliability thresholds, and
the augmented data were more balanced in classes. This novel CPSC provides
higher prediction accuracy and better reliability quantification, which can be
a reliable assistance in decision support.
