  The healthcare industry has been revolutionized by the convergence of
Artificial Intelligence of Medical Things (AIoMT), allowing advanced
data-driven solutions to improve healthcare systems. With the increasing
complexity of Artificial Intelligence (AI) models, the need for Explainable
Artificial Intelligence (XAI) techniques become paramount, particularly in the
medical domain, where transparent and interpretable decision-making becomes
crucial. Therefore, in this work, we leverage a custom XAI framework,
incorporating techniques such as Local Interpretable Model-Agnostic
Explanations (LIME), SHapley Additive exPlanations (SHAP), and
Gradient-weighted Class Activation Mapping (Grad-Cam), explicitly designed for
the domain of AIoMT. The proposed framework enhances the effectiveness of
strategic healthcare methods and aims to instill trust and promote
understanding in AI-driven medical applications. Moreover, we utilize a
majority voting technique that aggregates predictions from multiple
convolutional neural networks (CNNs) and leverages their collective
intelligence to make robust and accurate decisions in the healthcare system.
Building upon this decision-making process, we apply the XAI framework to brain
tumor detection as a use case demonstrating accurate and transparent diagnosis.
Evaluation results underscore the exceptional performance of the XAI framework,
achieving high precision, recall, and F1 scores with a training accuracy of 99%
and a validation accuracy of 98%. Combining advanced XAI techniques with
ensemble-based deep-learning (DL) methodologies allows for precise and reliable
brain tumor diagnoses as an application of AIoMT.
