  For many applications of probabilistic classifiers it is important that the
predicted confidence vectors reflect true probabilities (one says that the
classifier is calibrated). It has been shown that common models fail to satisfy
this property, making reliable methods for measuring and improving calibration
important tools. Unfortunately, obtaining these is far from trivial for
problems with many classes. We propose two techniques that can be used in
tandem. First, a reduced calibration method transforms the original problem
into a simpler one. We prove for several notions of calibration that solving
the reduced problem minimizes the corresponding notion of miscalibration in the
full problem, allowing the use of non-parametric recalibration methods that
fail in higher dimensions. Second, we propose class-wise calibration methods,
based on intuition building on a phenomenon called neural collapse and the
observation that most of the accurate classifiers found in practice can be
thought of as a union of K different functions which can be recalibrated
separately, one for each class. These typically out-perform their non
class-wise counterparts, especially for classifiers trained on imbalanced data
sets. Applying the two methods together results in class-wise reduced
calibration algorithms, which are powerful tools for reducing the prediction
and per-class calibration errors. We demonstrate our methods on real and
synthetic datasets and release all code as open source at
https://github.com/appliedAI-Initiative
