  Recent object tracking methods depend upon deep networks or convoluted
architectures. Most of those trackers can hardly meet real-time processing
requirements on mobile platforms with limited computing resources. In this
work, we introduce the Siamese Transformer Pyramid Network (SiamTPN), which
inherits the advantages from both CNN and Transformer architectures.
Specifically, we exploit the inherent feature pyramid of a lightweight network
(ShuffleNetV2) and reinforce it with a Transformer to construct a robust
target-specific appearance model. A centralized architecture with lateral cross
attention is developed for building augmented high-level feature maps. To avoid
the computation and memory intensity while fusing pyramid representations with
the Transformer, we further introduce the pooling attention module, which
significantly reduces memory and time complexity while improving the
robustness. Comprehensive experiments on both aerial and prevalent tracking
benchmarks achieve competitive results while operating at high speed,
demonstrating the effectiveness of SiamTPN. Moreover, our fastest variant
tracker operates over 30 Hz on a single CPU-core and obtaining an AUC score of
58.1% on the LaSOT dataset. Source codes are available at
https://github.com/RISCNYUAD/SiamTPNTracker
