  Siamese trackers turn tracking into similarity estimation between a template
and the candidate regions in the frame. Mathematically, one of the key
ingredients of success of the similarity function is translation equivariance.
Non-translation-equivariant architectures induce a positional bias during
training, so the location of the target will be hard to recover from the
feature space. In real life scenarios, objects undergoe various transformations
other than translation, such as rotation or scaling. Unless the model has an
internal mechanism to handle them, the similarity may degrade. In this paper,
we focus on scaling and we aim to equip the Siamese network with additional
built-in scale equivariance to capture the natural variations of the target a
priori. We develop the theory for scale-equivariant Siamese trackers, and
provide a simple recipe for how to make a wide range of existing trackers
scale-equivariant. We present SE-SiamFC, a scale-equivariant variant of SiamFC
built according to the recipe. We conduct experiments on OTB and VOT benchmarks
and on the synthetically generated T-MNIST and S-MNIST datasets. We demonstrate
that a built-in additional scale equivariance is useful for visual object
tracking.
