  Siamese network based trackers formulate tracking as convolutional feature
cross-correlation between target template and searching region. However,
Siamese trackers still have accuracy gap compared with state-of-the-art
algorithms and they cannot take advantage of feature from deep networks, such
as ResNet-50 or deeper. In this work we prove the core reason comes from the
lack of strict translation invariance. By comprehensive theoretical analysis
and experimental validations, we break this restriction through a simple yet
effective spatial aware sampling strategy and successfully train a
ResNet-driven Siamese tracker with significant performance gain. Moreover, we
propose a new model architecture to perform depth-wise and layer-wise
aggregations, which not only further improves the accuracy but also reduces the
model size. We conduct extensive ablation studies to demonstrate the
effectiveness of the proposed tracker, which obtains currently the best results
on four large tracking benchmarks, including OTB2015, VOT2018, UAV123, and
LaSOT. Our model will be released to facilitate further studies based on this
problem.
