  Although 3D point cloud classification neural network models have been widely
used, the in-depth interpretation of the activation of the neurons and layers
is still a challenge. We propose a novel approach, named Relevance Flow, to
interpret the hidden semantics of 3D point cloud classification neural
networks. It delivers the class Relevance to the activated neurons in the
intermediate layers in a back-propagation manner, and associates the activation
of neurons with the input points to visualize the hidden semantics of each
layer. Specially, we reveal that the 3D point cloud classification neural
network has learned the plane-level and part-level hidden semantics in the
intermediate layers, and utilize the normal and IoU to evaluate the consistency
of both levels' hidden semantics. Besides, by using the hidden semantics, we
generate the adversarial attack samples to attack 3D point cloud classifiers.
Experiments show that our proposed method reveals the hidden semantics of the
3D point cloud classification neural network on ModelNet40 and ShapeNet, which
can be used for the unsupervised point cloud part segmentation without labels
and attacking the 3D point cloud classifiers.
