  Explainability is often critical to the acceptable implementation of
artificial intelligence (AI). Nowhere is this more important than healthcare
where decision-making directly impacts patients and trust in AI systems is
essential. This trust is often built on the explanations and interpretations
the AI provides. Despite significant advancements in AI interpretability, there
remains the need for clear guidelines on when and to what extent explanations
are necessary in the medical context. We propose a novel categorization system
with four distinct classes of explanation necessity, guiding the level of
explanation required: patient or sample (local) level, cohort or dataset
(global) level, or both levels. We introduce a mathematical formulation that
distinguishes these categories and offers a practical framework for researchers
to determine the necessity and depth of explanations required in medical AI
applications. Three key factors are considered: the robustness of the
evaluation protocol, the variability of expert observations, and the
representation dimensionality of the application. In this perspective, we
address the question: When does an AI medical application need to be explained,
and at what level of detail?
