  Accurate probabilistic predictions are essential for optimal decision making.
While neural network miscalibration has been studied primarily in
classification, we investigate this in the less-explored domain of regression.
We conduct the largest empirical study to date to assess the probabilistic
calibration of neural networks. We also analyze the performance of
recalibration, conformal, and regularization methods to enhance probabilistic
calibration. Additionally, we introduce novel differentiable recalibration and
regularization methods, uncovering new insights into their effectiveness. Our
findings reveal that regularization methods offer a favorable tradeoff between
calibration and sharpness. Post-hoc methods exhibit superior probabilistic
calibration, which we attribute to the finite-sample coverage guarantee of
conformal prediction. Furthermore, we demonstrate that quantile recalibration
can be considered as a specific case of conformal prediction. Our study is
fully reproducible and implemented in a common code base for fair comparisons.
