  Brain-computer interface (BCI) systems facilitate unique communication
between humans and computers, benefiting severely disabled individuals. Despite
decades of research, BCIs are not fully integrated into clinical and commercial
settings. It's crucial to assess and explain BCI performance, offering clear
explanations for potential users to avoid frustration when it doesn't work as
expected. This work investigates the efficacy of different deep learning and
Riemannian geometry-based classification models in the context of motor imagery
(MI) based BCI using electroencephalography (EEG). We then propose an optimal
transport theory-based approach using earth mover's distance (EMD) to quantify
the comparison of the feature relevance map with the domain knowledge of
neuroscience. For this, we utilized explainable AI (XAI) techniques for
generating feature relevance in the spatial domain to identify important
channels for model outcomes. Three state-of-the-art models are implemented - 1)
Riemannian geometry-based classifier, 2) EEGNet, and 3) EEG Conformer, and the
observed trend in the model's accuracy across different architectures on the
dataset correlates with the proposed feature relevance metrics. The models with
diverse architectures perform significantly better when trained on channels
relevant to motor imagery than data-driven channel selection. This work focuses
attention on the necessity for interpretability and incorporating metrics
beyond accuracy, underscores the value of combining domain knowledge and
quantifying model interpretations with data-driven approaches in creating
reliable and robust Brain-Computer Interfaces (BCIs).
