  The training of deep neural networks (DNNs) always requires intensive
resources for both computation and data storage. Thus, DNNs cannot be
efficiently applied to mobile phones and embedded devices, which severely
limits their applicability in industrial applications. To address this issue,
we propose a novel encoding scheme using {-1, +1} to decompose quantized neural
networks (QNNs) into multi-branch binary networks, which can be efficiently
implemented by bitwise operations (i.e., xnor and bitcount) to achieve model
compression, computational acceleration, and resource saving. By using our
method, users can achieve different encoding precisions arbitrarily according
to their requirements and hardware resources. The proposed mechanism is highly
suitable for the use of FPGA and ASIC in terms of data storage and computation,
which provides a feasible idea for smart chips. We validate the effectiveness
of our method on large-scale image classification (e.g., ImageNet), object
detection, and semantic segmentation tasks. In particular, our method with
low-bit encoding can still achieve almost the same performance as its high-bit
counterparts.
