  The wide-spread adoption of representation learning technologies in clinical
decision making strongly emphasizes the need for characterizing model
reliability and enabling rigorous introspection of model behavior. While the
former need is often addressed by incorporating uncertainty quantification
strategies, the latter challenge is addressed using a broad class of
interpretability techniques. In this paper, we argue that these two objectives
are not necessarily disparate and propose to utilize prediction calibration to
meet both objectives. More specifically, our approach is comprised of a
calibration-driven learning method, which is also used to design an
interpretability technique based on counterfactual reasoning. Furthermore, we
introduce \textit{reliability plots}, a holistic evaluation mechanism for model
reliability. Using a lesion classification problem with dermoscopy images, we
demonstrate the effectiveness of our approach and infer interesting insights
about the model behavior.
