  The current definition of a Bayesian credible set cannot, in general, achieve
an arbitrarily preassigned credible level. This drawback is particularly acute
for classification problems, where there are only a finite number of achievable
credible levels. As a result, there is as of today no general way to construct
an exact credible set for classification. In this paper, we introduce a
generalized credible set that can achieve any preassigned credible level. The
key insight is a simple connection between the Bayesian highest posterior
density credible set and the Neyman--Pearson lemma, which, as far as we know,
hasn't been noticed before. Using this connection, we introduce a randomized
decision rule to fill the gaps among the discrete credible levels. Accompanying
this methodology, we also develop the Steering Wheel Plot to represent the
credible set, which is useful in visualizing the uncertainty in classification.
By developing the exact credible set for discrete parameters, we make the
theory of Bayesian inference more complete.
