  In this paper, we introduce a new hierarchical model for human action
recognition using body joint locations. Our model can categorize complex
actions in videos, and perform spatio-temporal annotations of the atomic
actions that compose the complex action being performed.That is, for each
atomic action, the model generates temporal action annotations by estimating
its starting and ending times, as well as, spatial annotations by inferring the
human body parts that are involved in executing the action. our model includes
three key novel properties: (i) it can be trained with no spatial supervision,
as it can automatically discover active body parts from temporal action
annotations only; (ii) it jointly learns flexible representations for motion
poselets and actionlets that encode the visual variability of body parts and
atomic actions; (iii) a mechanism to discard idle or non-informative body parts
which increases its robustness to common pose estimation errors. We evaluate
the performance of our method using multiple action recognition benchmarks. Our
model consistently outperforms baselines and state-of-the-art action
recognition methods.
