  Background: Pneumothorax is an acute thoracic disease caused by abnormal air
collection between the lungs and chest wall. To address the opaqueness often
associated with deep learning (DL) models, explainable artificial intelligence
(XAI) methods have been introduced to outline regions related to pneumothorax
diagnoses made by DL models. However, these explanations sometimes diverge from
actual lesion areas, highlighting the need for further improvement. Method: We
propose a template-guided approach to incorporate the clinical knowledge of
pneumothorax into model explanations generated by XAI methods, thereby
enhancing the quality of these explanations. Utilizing one lesion delineation
created by radiologists, our approach first generates a template that
represents potential areas of pneumothorax occurrence. This template is then
superimposed on model explanations to filter out extraneous explanations that
fall outside the template's boundaries. To validate its efficacy, we carried
out a comparative analysis of three XAI methods with and without our template
guidance when explaining two DL models in two real-world datasets. Results: The
proposed approach consistently improved baseline XAI methods across twelve
benchmark scenarios built on three XAI methods, two DL models, and two
datasets. The average incremental percentages, calculated by the performance
improvements over the baseline performance, were 97.8% in Intersection over
Union (IoU) and 94.1% in Dice Similarity Coefficient (DSC) when comparing model
explanations and ground-truth lesion areas. Conclusions: In the context of
pneumothorax diagnoses, we proposed a template-guided approach for improving AI
explanations. We anticipate that our template guidance will forge a fresh
approach to elucidating AI models by integrating clinical domain expertise.
