  Skeleton-based human action recognition has attracted a lot of research
attention during the past few years. Recent works attempted to utilize
recurrent neural networks to model the temporal dependencies between the 3D
positional configurations of human body joints for better analysis of human
activities in the skeletal data. The proposed work extends this idea to spatial
domain as well as temporal domain to better analyze the hidden sources of
action-related information within the human skeleton sequences in both of these
domains simultaneously. Based on the pictorial structure of Kinect's skeletal
data, an effective tree-structure based traversal framework is also proposed.
In order to deal with the noise in the skeletal data, a new gating mechanism
within LSTM module is introduced, with which the network can learn the
reliability of the sequential data and accordingly adjust the effect of the
input data on the updating procedure of the long-term context representation
stored in the unit's memory cell. Moreover, we introduce a novel multi-modal
feature fusion strategy within the LSTM unit in this paper. The comprehensive
experimental results on seven challenging benchmark datasets for human action
recognition demonstrate the effectiveness of the proposed method.
