  We propose and demonstrate an alternating Fourier and image domain filtering
approach for feature extraction as an efficient alternative to build a vision
backbone without using the computationally intensive attention. The performance
among the lightweight models reaches the state-of-the-art level on ImageNet-1K
classification, and improves downstream tasks on object detection and
segmentation consistently as well. Our approach also serves as a new tool to
compress vision transformers (ViTs).
