  We explore the black-box adversarial attack on video recognition models.
Attacks are only performed on selected key regions and key frames to reduce the
high computation cost of searching adversarial perturbations on a video due to
its high dimensionality. To select key frames, one way is to use heuristic
algorithms to evaluate the importance of each frame and choose the essential
ones. However, it is time inefficient on sorting and searching. In order to
speed up the attack process, we propose a reinforcement learning based frame
selection strategy. Specifically, the agent explores the difference between the
original class and the target class of videos to make selection decisions. It
receives rewards from threat models which indicate the quality of the
decisions. Besides, we also use saliency detection to select key regions and
only estimate the sign of gradient instead of the gradient itself in zeroth
order optimization to further boost the attack process. We can use the trained
model directly in the untargeted attack or with little fine-tune in the
targeted attack, which saves computation time. A range of empirical results on
real datasets demonstrate the effectiveness and efficiency of the proposed
method.
