  We introduce CoTracker, a transformer-based model that tracks dense points in
a frame jointly across a video sequence. This differs from most existing
state-of-the-art approaches that track points independently, ignoring their
correlation. We show that joint tracking results in a significantly higher
tracking accuracy and robustness. We also provide several technical
innovations, including the concept of virtual tracks, which allows CoTracker to
track 70k points jointly and simultaneously. Furthermore, CoTracker operates
causally on short windows (hence, it is suitable for online tasks), but is
trained by unrolling the windows across longer video sequences, which enables
and significantly improves long-term tracking. We demonstrate qualitatively
impressive tracking results, where points can be tracked for a long time even
when they are occluded or leave the field of view. Quantitatively, CoTracker
outperforms all recent trackers on standard benchmarks, often by a substantial
margin.
