  It's common for current methods in skeleton-based action recognition to
mainly consider capturing long-term temporal dependencies as skeleton sequences
are typically long (>128 frames), which forms a challenging problem for
previous approaches. In such conditions, short-term dependencies are few
formally considered, which are critical for classifying similar actions. Most
current approaches are consisted of interleaving spatial-only modules and
temporal-only modules, where direct information flow among joints in adjacent
frames are hindered, thus inferior to capture short-term motion and distinguish
similar action pairs. To handle this limitation, we propose a general
framework, coined as STGAT, to model cross-spacetime information flow. It
equips the spatial-only modules with spatial-temporal modeling for regional
perception. While STGAT is theoretically effective for spatial-temporal
modeling, we propose three simple modules to reduce local spatial-temporal
feature redundancy and further release the potential of STGAT, which (1) narrow
the scope of self-attention mechanism, (2) dynamically weight joints along
temporal dimension, and (3) separate subtle motion from static features,
respectively. As a robust feature extractor, STGAT generalizes better upon
classifying similar actions than previous methods, witnessed by both
qualitative and quantitative results. STGAT achieves state-of-the-art
performance on three large-scale datasets: NTU RGB+D 60, NTU RGB+D 120, and
Kinetics Skeleton 400. Code is released.
