  Recently, adversarial attacks have been applied in visual object tracking to
deceive deep trackers by injecting imperceptible perturbations into video
frames. However, previous work only generates the video-specific perturbations,
which restricts its application scenarios. In addition, existing attacks are
difficult to implement in reality due to the real-time of tracking and the
re-initialization mechanism. To address these issues, we propose an offline
universal adversarial attack called Efficient Universal Shuffle Attack. It
takes only one perturbation to cause the tracker malfunction on all videos. To
improve the computational efficiency and attack performance, we propose a
greedy gradient strategy and a triple loss to efficiently capture and attack
model-specific feature representations through the gradients. Experimental
results show that EUSA can significantly reduce the performance of
state-of-the-art trackers on OTB2015 and VOT2018.
