  Existing methods of privacy-preserving action recognition (PPAR) mainly focus
on frame-level (spatial) privacy removal through 2D CNNs. Unfortunately, they
have two major drawbacks. First, they may compromise temporal dynamics in input
videos, which are critical for accurate action recognition. Second, they are
vulnerable to practical attacking scenarios where attackers probe for privacy
from an entire video rather than individual frames. To address these issues, we
propose a novel framework STPrivacy to perform video-level PPAR. For the first
time, we introduce vision Transformers into PPAR by treating a video as a
tubelet sequence, and accordingly design two complementary mechanisms, i.e.,
sparsification and anonymization, to remove privacy from a spatio-temporal
perspective. In specific, our privacy sparsification mechanism applies adaptive
token selection to abandon action-irrelevant tubelets. Then, our anonymization
mechanism implicitly manipulates the remaining action-tubelets to erase privacy
in the embedding space through adversarial learning. These mechanisms provide
significant advantages in terms of privacy preservation for human eyes and
action-privacy trade-off adjustment during deployment. We additionally
contribute the first two large-scale PPAR benchmarks, VP-HMDB51 and VP-UCF101,
to the community. Extensive evaluations on them, as well as two other tasks,
validate the effectiveness and generalization capability of our framework.
