  Explaining neural network models is a challenging task that remains unsolved
in its entirety to this day. This is especially true for high dimensional and
complex data. With the present work, we introduce two notions for conceptual
views of a neural network, specifically a many-valued and a symbolic view. Both
provide novel analysis methods to enable a human AI analyst to grasp deeper
insights into the knowledge that is captured by the neurons of a network. We
test the conceptual expressivity of our novel views through different
experiments on the ImageNet and Fruit-360 data sets. Furthermore, we show to
which extent the views allow to quantify the conceptual similarity of different
learning architectures. Finally, we demonstrate how conceptual views can be
applied for abductive learning of human comprehensible rules from neurons. In
summary, with our work, we contribute to the most relevant task of globally
explaining neural networks models.
