  Although synthetic training data has been shown to be beneficial for tasks
such as human pose estimation, its use for RGB human action recognition is
relatively unexplored. Our goal in this work is to answer the question whether
synthetic humans can improve the performance of human action recognition, with
a particular focus on generalization to unseen viewpoints. We make use of the
recent advances in monocular 3D human body reconstruction from real action
sequences to automatically render synthetic training videos for the action
labels. We make the following contributions: (i) we investigate the extent of
variations and augmentations that are beneficial to improving performance at
new viewpoints. We consider changes in body shape and clothing for individuals,
as well as more action relevant augmentations such as non-uniform frame
sampling, and interpolating between the motion of individuals performing the
same action; (ii) We introduce a new data generation methodology, SURREACT,
that allows training of spatio-temporal CNNs for action classification; (iii)
We substantially improve the state-of-the-art action recognition performance on
the NTU RGB+D and UESTC standard human action multi-view benchmarks; Finally,
(iv) we extend the augmentation approach to in-the-wild videos from a subset of
the Kinetics dataset to investigate the case when only one-shot training data
is available, and demonstrate improvements in this case as well.
