  Adversarial attacks on video recognition models have been explored recently.
However, most existing works treat each video frame equally and ignore their
temporal interactions. To overcome this drawback, a few methods try to select
some key frames and then perform attacks based on them. Unfortunately, their
selection strategy is independent of the attacking step, therefore the
resulting performance is limited. Instead, we argue the frame selection phase
is closely relevant with the attacking phase. The key frames should be adjusted
according to the attacking results. For that, we formulate the black-box video
attacks into a Reinforcement Learning (RL) framework. Specifically, the
environment in RL is set as the recognition model, and the agent in RL plays
the role of frame selecting. By continuously querying the recognition models
and receiving the attacking feedback, the agent gradually adjusts its frame
selection strategy and adversarial perturbations become smaller and smaller. We
conduct a series of experiments with two mainstream video recognition models:
C3D and LRCN on the public UCF-101 and HMDB-51 datasets. The results
demonstrate that the proposed method can significantly reduce the adversarial
perturbations with efficient query times.
