  Recently, deep learning approach has achieved promising results in various
fields of computer vision. In this paper, a new framework called Hierarchical
Depth Motion Maps (HDMM) + 3 Channel Deep Convolutional Neural Networks
(3ConvNets) is proposed for human action recognition using depth map sequences.
Firstly, we rotate the original depth data in 3D pointclouds to mimic the
rotation of cameras, so that our algorithms can handle view variant cases.
Secondly, in order to effectively extract the body shape and motion
information, we generate weighted depth motion maps (DMM) at several temporal
scales, referred to as Hierarchical Depth Motion Maps (HDMM). Then, three
channels of ConvNets are trained on the HDMMs from three projected orthogonal
planes separately. The proposed algorithms are evaluated on MSRAction3D,
MSRAction3DExt, UTKinect-Action and MSRDailyActivity3D datasets respectively.
We also combine the last three datasets into a larger one (called Combined
Dataset) and test the proposed method on it. The results show that our approach
can achieve state-of-the-art results on the individual datasets and without
dramatical performance degradation on the Combined Dataset.
