  Human actions involve complex pose variations and their 2D projections can be
highly ambiguous. Thus 3D spatio-temporal or 4D (i.e., 3D+T) human skeletons,
which are photometric and viewpoint invariant, are an excellent alternative to
2D+T skeletons/pixels to improve action recognition accuracy. This paper
proposes a new 4D dataset HAA4D which consists of more than 3,300 RGB videos in
300 human atomic action classes. HAA4D is clean, diverse, class-balanced where
each class is viewpoint-balanced with the use of 4D skeletons, in which as few
as one 4D skeleton per class is sufficient for training a deep recognition
model. Further, the choice of atomic actions makes annotation even easier,
because each video clip lasts for only a few seconds. All training and testing
3D skeletons in HAA4D are globally aligned, using a deep alignment model to the
same global space, making each skeleton face the negative z-direction. Such
alignment makes matching skeletons more stable by reducing intraclass
variations and thus with fewer training samples per class needed for action
recognition. Given the high diversity and skeletal alignment in HAA4D, we
construct the first baseline few-shot 4D human atomic action recognition
network without bells and whistles, which produces comparable or higher
performance than relevant state-of-the-art techniques relying on embedded space
encoding without explicit skeletal alignment, using the same small number of
training samples of unseen classes.
