  Action recognition from well-segmented 3D skeleton video has been intensively
studied. However, due to the difficulty in representing the 3D skeleton video
and the lack of training data, action detection from streaming 3D skeleton
video still lags far behind its recognition counterpart and image based object
detection. In this paper, we propose a novel approach for this problem, which
leverages both effective skeleton video encoding and deep regression based
object detection from images. Our framework consists of two parts:
skeleton-based video image mapping, which encodes a skeleton video to a color
image in a temporal preserving way, and an end-to-end trainable fast skeleton
action detector (Skeleton Boxes) based on image detection. Experimental results
on the latest and largest PKU-MMD benchmark dataset demonstrate that our method
outperforms the state-of-the-art methods with a large margin. We believe our
idea would inspire and benefit future research in this important area.
