  This paper studies the problem of designing compact binary architectures for
vision multi-layer perceptrons (MLPs). We provide extensive analysis on the
difficulty of binarizing vision MLPs and find that previous binarization
methods perform poorly due to limited capacity of binary MLPs. In contrast with
the traditional CNNs that utilizing convolutional operations with large kernel
size, fully-connected (FC) layers in MLPs can be treated as convolutional
layers with kernel size $1\times1$. Thus, the representation ability of the FC
layers will be limited when being binarized, and places restrictions on the
capability of spatial mixing and channel mixing on the intermediate features.
To this end, we propose to improve the performance of binary MLP (BiMLP) model
by enriching the representation ability of binary FC layers. We design a novel
binary block that contains multiple branches to merge a series of outputs from
the same stage, and also a universal shortcut connection that encourages the
information flow from the previous stage. The downsampling layers are also
carefully designed to reduce the computational complexity while maintaining the
classification performance. Experimental results on benchmark dataset
ImageNet-1k demonstrate the effectiveness of the proposed BiMLP models, which
achieve state-of-the-art accuracy compared to prior binary CNNs. The MindSpore
code is available at
\url{https://gitee.com/mindspore/models/tree/master/research/cv/BiMLP}.
