  Object trackers with transformer backbones have achieved robust performance
on visual object tracking datasets. However, the adversarial robustness of
these trackers has not been well studied in the literature. Due to the backbone
differences, the adversarial white-box attacks proposed for object tracking are
not transferable to all types of trackers. For instance, transformer trackers
such as MixFormerM still function well after black-box attacks, especially in
predicting the object binary masks. We are proposing a novel white-box attack
named TrackPGD, which relies on the predicted object binary mask to attack the
robust transformer trackers. That new attack focuses on annotation masks by
adapting the well-known SegPGD segmentation attack, allowing to successfully
conduct the white-box attack on trackers relying on transformer backbones. The
experimental results indicate that the TrackPGD is able to effectively attack
transformer-based trackers such as MixFormerM, OSTrackSTS, and TransT-SEG on
several tracking datasets.
