  This paper proposes a new neural network based on SPD manifold learning for
skeleton-based hand gesture recognition. Given the stream of hand's joint
positions, our approach combines two aggregation processes on respectively
spatial and temporal domains. The pipeline of our network architecture consists
in three main stages. The first stage is based on a convolutional layer to
increase the discriminative power of learned features. The second stage relies
on different architectures for spatial and temporal Gaussian aggregation of
joint features. The third stage learns a final SPD matrix from skeletal data. A
new type of layer is proposed for the third stage, based on a variant of
stochastic gradient descent on Stiefel manifolds. The proposed network is
validated on two challenging datasets and shows state-of-the-art accuracies on
both datasets.
