  The concept of refinement from probability elicitation is considered for
proper scoring rules. Taking directions from the axioms of probability,
refinement is further clarified using a Hilbert space interpretation and
reformulated into the underlying data distribution setting where connections to
maximal marginal diversity and conditional entropy are considered and used to
derive measures that provide arbitrarily tight bounds on the Bayes error.
Refinement is also reformulated into the classifier output setting and its
connections to calibrated classifiers and proper margin losses are established.
