  Extensive work has demonstrated the effectiveness of Vision Transformers. The
plain Vision Transformer tends to obtain multi-scale features by selecting
fixed layers, or the last layer of features aiming to achieve higher
performance in dense prediction tasks. However, this selection is often based
on manual operation. And different samples often exhibit different features at
different layers (e.g., edge, structure, texture, detail, etc.). This requires
us to seek a dynamic adaptive fusion method to filter different layer features.
In this paper, unlike previous encoder and decoder work, we design a neck
network for adaptive fusion and feature selection, called ViTController. We
validate the effectiveness of our method on different datasets and models and
surpass previous state-of-the-art methods. Finally, our method can also be used
as a plug-in module and inserted into different networks.
