  Although Vision Transformer (ViT) has achieved significant success in
computer vision, it does not perform well in dense prediction tasks due to the
lack of inner-patch information interaction and the limited diversity of
feature scale. Most existing studies are devoted to designing vision-specific
transformers to solve the above problems, which introduce additional
pre-training costs. Therefore, we present a plain, pre-training-free, and
feature-enhanced ViT backbone with Convolutional Multi-scale feature
interaction, named ViT-CoMer, which facilitates bidirectional interaction
between CNN and transformer. Compared to the state-of-the-art, ViT-CoMer has
the following advantages: (1) We inject spatial pyramid multi-receptive field
convolutional features into the ViT architecture, which effectively alleviates
the problems of limited local information interaction and single-feature
representation in ViT. (2) We propose a simple and efficient CNN-Transformer
bidirectional fusion interaction module that performs multi-scale fusion across
hierarchical features, which is beneficial for handling dense prediction tasks.
(3) We evaluate the performance of ViT-CoMer across various dense prediction
tasks, different frameworks, and multiple advanced pre-training. Notably, our
ViT-CoMer-L achieves 64.3% AP on COCO val2017 without extra training data, and
62.1% mIoU on ADE20K val, both of which are comparable to state-of-the-art
methods. We hope ViT-CoMer can serve as a new backbone for dense prediction
tasks to facilitate future research. The code will be released at
https://github.com/Traffic-X/ViT-CoMer.
