  The advancement of visual tracking has continuously been brought by deep
learning models. Typically, supervised learning is employed to train these
models with expensive labeled data. In order to reduce the workload of manual
annotations and learn to track arbitrary objects, we propose an unsupervised
learning method for visual tracking. The motivation of our unsupervised
learning is that a robust tracker should be effective in bidirectional
tracking. Specifically, the tracker is able to forward localize a target object
in successive frames and backtrace to its initial position in the first frame.
Based on such a motivation, in the training process, we measure the consistency
between forward and backward trajectories to learn a robust tracker from
scratch merely using unlabeled videos. We build our framework on a Siamese
correlation filter network, and propose a multi-frame validation scheme and a
cost-sensitive loss to facilitate unsupervised learning. Without bells and
whistles, the proposed unsupervised tracker achieves the baseline accuracy as
classic fully supervised trackers while achieving a real-time speed.
Furthermore, our unsupervised framework exhibits a potential in leveraging more
unlabeled or weakly labeled data to further improve the tracking accuracy.
