  In this paper, we focus on mean-field variational Bayesian Neural Networks
(BNNs) and explore the representation capacity of such BNNs by investigating
which types of concepts are less likely to be encoded by the BNN. It has been
observed and studied that a relatively small set of interactive concepts
usually emerge in the knowledge representation of a sufficiently-trained neural
network, and such concepts can faithfully explain the network output. Based on
this, our study proves that compared to standard deep neural networks (DNNs),
it is less likely for BNNs to encode complex concepts. Experiments verify our
theoretical proofs. Note that the tendency to encode less complex concepts does
not necessarily imply weak representation power, considering that complex
concepts exhibit low generalization power and high adversarial vulnerability.
The code is available at https://github.com/sjtu-xai-lab/BNN-concepts.
