  Skeleton-based action recognition has attracted increasing attention due to
its strong adaptability to dynamic circumstances and potential for broad
applications such as autonomous and anonymous surveillance. With the help of
deep learning techniques, it has also witnessed substantial progress and
currently achieved around 90\% accuracy in benign environment. On the other
hand, research on the vulnerability of skeleton-based action recognition under
different adversarial settings remains scant, which may raise security concerns
about deploying such techniques into real-world systems. However, filling this
research gap is challenging due to the unique physical constraints of skeletons
and human actions. In this paper, we attempt to conduct a thorough study
towards understanding the adversarial vulnerability of skeleton-based action
recognition. We first formulate generation of adversarial skeleton actions as a
constrained optimization problem by representing or approximating the
physiological and physical constraints with mathematical formulations. Since
the primal optimization problem with equality constraints is intractable, we
propose to solve it by optimizing its unconstrained dual problem using ADMM. We
then specify an efficient plug-in defense, inspired by recent theories and
empirical observations, against the adversarial skeleton actions. Extensive
evaluations demonstrate the effectiveness of the attack and defense method
under different settings.
