  Fair calibration is a widely desirable fairness criteria in risk prediction
contexts. One way to measure and achieve fair calibration is with
multicalibration. Multicalibration constrains calibration error among
flexibly-defined subpopulations while maintaining overall calibration. However,
multicalibrated models can exhibit a higher percent calibration error among
groups with lower base rates than groups with higher base rates. As a result,
it is possible for a decision-maker to learn to trust or distrust model
predictions for specific groups. To alleviate this, we propose
\emph{proportional multicalibration}, a criteria that constrains the percent
calibration error among groups and within prediction bins. We prove that
satisfying proportional multicalibration bounds a model's multicalibration as
well its \emph{differential calibration}, a fairness criteria that directly
measures how closely a model approximates sufficiency. Therefore,
proportionally calibrated models limit the ability of decision makers to
distinguish between model performance on different patient groups, which may
make the models more trustworthy in practice. We provide an efficient algorithm
for post-processing risk prediction models for proportional multicalibration
and evaluate it empirically. We conduct simulation studies and investigate a
real-world application of PMC-postprocessing to prediction of emergency
department patient admissions. We observe that proportional multicalibration is
a promising criteria for controlling simultaneous measures of calibration
fairness of a model over intersectional groups with virtually no cost in terms
of classification performance.
