  Human action recognition is a challenging problem, particularly when there is
high variability in factors such as subject appearance, backgrounds and
viewpoint. While deep neural networks (DNNs) have been shown to perform well on
action recognition tasks, they typically require large amounts of high-quality
labeled data to achieve robust performance across a variety of conditions.
Synthetic data has shown promise as a way to avoid the substantial costs and
potential ethical concerns associated with collecting and labeling enormous
amounts of data in the real-world. However, synthetic data may differ from real
data in important ways. This phenomenon, known as \textit{domain shift}, can
limit the utility of synthetic data in robotics applications. To mitigate the
effects of domain shift, substantial effort is being dedicated to the
development of domain adaptation (DA) techniques. Yet, much remains to be
understood about how best to develop these techniques. In this paper, we
introduce a new dataset called Robot Control Gestures (RoCoG-v2). The dataset
is composed of both real and synthetic videos from seven gesture classes, and
is intended to support the study of synthetic-to-real domain shift for
video-based action recognition. Our work expands upon existing datasets by
focusing the action classes on gestures for human-robot teaming, as well as by
enabling investigation of domain shift in both ground and aerial views. We
present baseline results using state-of-the-art action recognition and domain
adaptation algorithms and offer initial insight on tackling the
synthetic-to-real and ground-to-air domain shifts.
