  Visual object tracking is an important task that requires the tracker to find
the objects quickly and accurately. The existing state-ofthe-art object
trackers, i.e., Siamese based trackers, use DNNs to attain high accuracy.
However, the robustness of visual tracking models is seldom explored. In this
paper, we analyze the weakness of object trackers based on the Siamese network
and then extend adversarial examples to visual object tracking. We present an
end-to-end network FAN (Fast Attack Network) that uses a novel drift loss
combined with the embedded feature loss to attack the Siamese network based
trackers. Under a single GPU, FAN is efficient in the training speed and has a
strong attack performance. The FAN can generate an adversarial example at 10ms,
achieve effective targeted attack (at least 40% drop rate on OTB) and
untargeted attack (at least 70% drop rate on OTB).
