  Skeleton data is of low dimension. However, there is a trend of using very
deep and complicated feedforward neural networks to model the skeleton sequence
without considering the complexity in recent year. In this paper, a simple yet
effective multi-scale semantics-guided neural network (MS-SGN) is proposed for
skeleton-based action recognition. We explicitly introduce the high level
semantics of joints (joint type and frame index) into the network to enhance
the feature representation capability of joints. Moreover, a multi-scale
strategy is proposed to be robust to the temporal scale variations. In
addition, we exploit the relationship of joints hierarchically through two
modules, i.e., a joint-level module for modeling the correlations of joints in
the same frame and a frame-level module for modeling the temporal dependencies
of frames. With an order of magnitude smaller model size than most previous
methods, MSSGN achieves the state-of-the-art performance on the NTU60, NTU120,
and SYSU datasets.
