  Theoretical guarantees for causal inference using propensity scores are
partly based on the scores behaving like conditional probabilities. However,
scores between zero and one, especially when outputted by flexible statistical
estimators, do not necessarily behave like probabilities. We perform a
simulation study to assess the error in estimating the average treatment effect
before and after applying a simple and well-established post-processing method
to calibrate the propensity scores. We find that post-calibration reduces the
error in effect estimation for expressive uncalibrated statistical estimators,
and that this improvement is not mediated by better balancing. The larger the
initial lack of calibration, the larger the improvement in effect estimation,
with the effect on already-calibrated estimators being very small. Given the
improvement in effect estimation and that post-calibration is computationally
cheap, we recommend it will be adopted when modelling propensity scores with
expressive models.
