  This work studies feature representations for dense label propagation in
video, with a focus on recently proposed methods that learn video
correspondence using self-supervised signals such as colorization or temporal
cycle consistency. In the literature, these methods have been evaluated with an
array of inconsistent settings, making it difficult to discern trends or
compare performance fairly. Starting with a unified formulation of the label
propagation algorithm that encompasses most existing variations, we
systematically study the impact of important implementation factors in feature
extraction and label propagation. Along the way, we report the accuracies of
properly tuned supervised and unsupervised still image baselines, which are
higher than those found in previous works. We also demonstrate that augmenting
video-based correspondence cues with still-image-based ones can further improve
performance. We then attempt a fair comparison of recent video-based methods on
the DAVIS benchmark, showing convergence of best methods to performance levels
near our strong ImageNet baseline, despite the usage of a variety of
specialized video-based losses and training particulars. Additional comparisons
on JHMDB and VIP datasets confirm the similar performance of current methods.
We hope that this study will help to improve evaluation practices and better
inform future research directions in temporal correspondence.
