  Online action detection (OAD) is challenging since 1) robust yet
computationally expensive features cannot be straightforwardly used due to the
real-time processing requirements and 2) the localization and classification of
actions have to be performed even before they are fully observed. We propose a
new random forest (RF)-based online action detection framework that addresses
these challenges. Our algorithm uses computationally efficient skeletal joint
features. High accuracy is achieved by using robust convolutional neural
network (CNN)-based features which are extracted from the raw RGBD images, plus
the temporal relationships between the current frame of interest, and the past
and future frames. While these high-quality features are not available in
real-time testing scenario, we demonstrate that they can be effectively
exploited in training RF classifiers: We use these spatio-temporal contexts to
craft RF's new split functions improving RFs' leaf node statistics. Experiments
with challenging MSRAction3D, G3D, and OAD datasets demonstrate that our
algorithm significantly improves the accuracy over the state-of-the-art online
action detection algorithms while achieving the real-time efficiency of
existing skeleton-based RF classifiers.
