  Despite their simpler information fusion designs compared with Vision
Transformers and Convolutional Neural Networks, Vision MLP architectures have
demonstrated strong performance and high data efficiency in recent research.
However, existing works such as CycleMLP and Vision Permutator typically model
spatial information in equal-size spatial regions and do not consider
cross-scale spatial interactions. Further, their token mixers only model 1- or
2-axis correlations, avoiding 3-axis spatial-channel mixing due to its
computational demands. We therefore propose CS-Mixer, a hierarchical Vision MLP
that learns dynamic low-rank transformations for spatial-channel mixing through
cross-scale local and global aggregation. The proposed methodology achieves
competitive results on popular image recognition benchmarks without incurring
substantially more compute. Our largest model, CS-Mixer-L, reaches 83.2% top-1
accuracy on ImageNet-1k with 13.7 GFLOPs and 94 M parameters.
