  Interpretable deep learning is a fundamental building block towards safer AI,
especially when the deployment possibilities of deep learning-based
computer-aided medical diagnostic systems are so eminent. However, without a
computational formulation of black-box interpretation, general interpretability
research rely heavily on subjective bias. Clear decision structure of the
medical diagnostics lets us approximate the decision process of a radiologist
as a model - removed from subjective bias. We define the process of
interpretation as a finite communication between a known model and a black-box
model to optimally map the black box's decision process in the known model.
Consequently, we define interpretability as maximal information gain over the
initial uncertainty about the black-box's decision within finite communication.
We relax this definition based on the observation that diagnostic
interpretation is typically achieved by a process of minimal querying. We
derive an algorithm to calculate diagnostic interpretability. The usual
question of accuracy-interpretability tradeoff, i.e. whether a black-box
model's prediction accuracy is dependent on its ability to be interpreted by a
known source model, does not arise in this theory. With multiple example
simulation experiments of various complexity levels, we demonstrate the working
of such a theoretical model in synthetic supervised classification scenarios.
