  Recently, Vision Transformers (ViT), with the self-attention (SA) as the de
facto ingredients, have demonstrated great potential in the computer vision
community. For the sake of trade-off between efficiency and performance, a
group of works merely perform SA operation within local patches, whereas the
global contextual information is abandoned, which would be indispensable for
visual recognition tasks. To solve the issue, the subsequent global-local ViTs
take a stab at marrying local SA with global one in parallel or alternative way
in the model. Nevertheless, the exhaustively combined local and global context
may exist redundancy for various visual data, and the receptive field within
each layer is fixed. Alternatively, a more graceful way is that global and
local context can adaptively contribute per se to accommodate different visual
data. To achieve this goal, we in this paper propose a novel ViT architecture,
termed NomMer, which can dynamically Nominate the synergistic global-local
context in vision transforMer. By investigating the working pattern of our
proposed NomMer, we further explore what context information is focused.
Beneficial from this "dynamic nomination" mechanism, without bells and
whistles, the NomMer can not only achieve 84.5% Top-1 classification accuracy
on ImageNet with only 73M parameters, but also show promising performance on
dense prediction tasks, i.e., object detection and semantic segmentation. The
code and models will be made publicly available at
https://github.com/TencentYoutuResearch/VisualRecognition-NomMer
