  Variations of target appearance such as deformations, illumination variance,
occlusion, etc., are the major challenges of visual object tracking that
negatively impact the performance of a tracker. An effective method to tackle
these challenges is template update, which updates the template to reflect the
change of appearance in the target object during tracking. However, with
template updates, inadequate quality of new templates or inappropriate timing
of updates may induce a model drift problem, which severely degrades the
tracking performance. Here, we propose BackTrack, a robust and reliable method
to quantify the confidence of the candidate template by backward tracking it on
the past frames. Based on the confidence score of candidates from BackTrack, we
can update the template with a reliable candidate at the right time while
rejecting unreliable candidates. BackTrack is a generic template update scheme
and is applicable to any template-based trackers. Extensive experiments on
various tracking benchmarks verify the effectiveness of BackTrack over existing
template update algorithms, as it achieves SOTA performance on various tracking
benchmarks.
