  Fisheye cameras suffer from image distortion while having a large field of
view(LFOV). And this fact leads to poor performance on some fisheye vision
tasks. One of the solutions is to optimize the current vision algorithm for
fisheye images. However, most of the CNN-based methods and the
Transformer-based methods lack the capability of leveraging distortion
information efficiently. In this work, we propose a novel patch embedding
method called Sector Patch Embedding(SPE), conforming to the distortion pattern
of the fisheye image. Furthermore, we put forward a synthetic fisheye dataset
based on the ImageNet-1K and explore the performance of several Transformer
models on the dataset. The classification top-1 accuracy of ViT and PVT is
improved by 0.75% and 2.8% with SPE respectively. The experiments show that the
proposed sector patch embedding method can better perceive distortion and
extract features on the fisheye images. Our method can be easily adopted to
other Transformer-based models. Source code is at
https://github.com/IN2-ViAUn/Sector-Patch-Embedding.
