  Current Siamese-based trackers mainly formulate the visual tracking into two
independent subtasks, including classification and localization. They learn the
classification subnetwork by processing each sample separately and neglect the
relationship among positive and negative samples. Moreover, such tracking
paradigm takes only the classification confidence of proposals for the final
prediction, which may yield the misalignment between classification and
localization. To resolve these issues, this paper proposes a ranking-based
optimization algorithm to explore the relationship among different proposals.
To this end, we introduce two ranking losses, including the classification one
and the IoU-guided one, as optimization constraints. The classification ranking
loss can ensure that positive samples rank higher than hard negative ones,
i.e., distractors, so that the trackers can select the foreground samples
successfully without being fooled by the distractors. The IoU-guided ranking
loss aims to align classification confidence scores with the Intersection over
Union(IoU) of the corresponding localization prediction for positive samples,
enabling the well-localized prediction to be represented by high classification
confidence. Specifically, the proposed two ranking losses are compatible with
most Siamese trackers and incur no additional computation for inference.
Extensive experiments on seven tracking benchmarks, including OTB100, UAV123,
TC128, VOT2016, NFS30, GOT-10k and LaSOT, demonstrate the effectiveness of the
proposed ranking-based optimization algorithm. The code and raw results are
available at https://github.com/sansanfree/RBO.
