  Human motion recognition is one of the most important branches of
human-centered research activities. In recent years, motion recognition based
on RGB-D data has attracted much attention. Along with the development in
artificial intelligence, deep learning techniques have gained remarkable
success in computer vision. In particular, convolutional neural networks (CNN)
have achieved great success for image-based tasks, and recurrent neural
networks (RNN) are renowned for sequence-based problems. Specifically, deep
learning methods based on the CNN and RNN architectures have been adopted for
motion recognition using RGB-D data. In this paper, a detailed overview of
recent advances in RGB-D-based motion recognition is presented. The reviewed
methods are broadly categorized into four groups, depending on the modality
adopted for recognition: RGB-based, depth-based, skeleton-based and
RGB+D-based. As a survey focused on the application of deep learning to
RGB-D-based motion recognition, we explicitly discuss the advantages and
limitations of existing techniques. Particularly, we highlighted the methods of
encoding spatial-temporal-structural information inherent in video sequence,
and discuss potential directions for future research.
