  Graph convolutional networks have been widely used for skeleton-based action
recognition due to their excellent modeling ability of non-Euclidean data. As
the graph convolution is a local operation, it can only utilize the short-range
joint dependencies and short-term trajectory but fails to directly model the
distant joints relations and long-range temporal information that are vital to
distinguishing various actions. To solve this problem, we present a multi-scale
spatial graph convolution (MS-GC) module and a multi-scale temporal graph
convolution (MT-GC) module to enrich the receptive field of the model in
spatial and temporal dimensions. Concretely, the MS-GC and MT-GC modules
decompose the corresponding local graph convolution into a set of sub-graph
convolution, forming a hierarchical residual architecture. Without introducing
additional parameters, the features will be processed with a series of
sub-graph convolutions, and each node could complete multiple spatial and
temporal aggregations with its neighborhoods. The final equivalent receptive
field is accordingly enlarged, which is capable of capturing both short- and
long-range dependencies in spatial and temporal domains. By coupling these two
modules as a basic block, we further propose a multi-scale spatial temporal
graph convolutional network (MST-GCN), which stacks multiple blocks to learn
effective motion representations for action recognition. The proposed MST-GCN
achieves remarkable performance on three challenging benchmark datasets, NTU
RGB+D, NTU-120 RGB+D and Kinetics-Skeleton, for skeleton-based action
recognition.
