  The interpretability of deep neural networks has attracted increasing
attention in recent years, and several methods have been created to interpret
the "black box" model. Fundamental limitations remain, however, that impede the
pace of understanding the networks, especially the extraction of understandable
semantic space. In this work, we introduce the framework of semantic
explainable AI (S-XAI), which utilizes row-centered principal component
analysis to obtain the common traits from the best combination of superpixels
discovered by a genetic algorithm, and extracts understandable semantic spaces
on the basis of discovered semantically sensitive neurons and visualization
techniques. Statistical interpretation of the semantic space is also provided,
and the concept of semantic probability is proposed for the first time. Our
experimental results demonstrate that S-XAI is effective in providing a
semantic interpretation for the CNN, and offers broad usage, including
trustworthiness assessment and semantic sample searching.
