  In the real world, camera-based application systems can face many challenges,
including environmental factors and distribution shift. In this paper, we
investigate how pose and shadow impact a classifier's performance, using the
specific application of handwashing action recognition. To accomplish this, we
generate synthetic data with desired variations to introduce controlled
distribution shift. Using our synthetic dataset, we define a classifier's
breakdown points to be where the system's performance starts to degrade
sharply, and we show these are heavily impacted by pose and shadow conditions.
In particular, heavier and larger shadows create earlier breakdown points.
Also, it is intriguing to observe model accuracy drop to almost zero with
bigger changes in pose. Moreover, we propose a simple mitigation strategy for
pose-induced breakdown points by utilizing additional training data from
non-canonical poses. Results show that the optimal choices of additional
training poses are those with moderate deviations from the canonical poses with
50-60 degrees of rotation.
